Alright, so a couple of weeks ago I stumbled upon Gimp Portable 2.4.4. I've been using Gimp 126.96.36.199 Portable for sometime on my flash drive at school, but it wasn't as flexible as the newer one, so yeah I'm glad they finally updated it. :D
Click here to get Gimp Portable 2.4.4
Try it out.