tFUnK,
I wrote a short article about this very topic a while back. Please take a read:
http://greatnexus.com/bike/a/7cheapways.php
In term of upgrading just to save weight, I think you will be better off replacing parts as they break. Depending on your bike, you can easily blow a few hundred dollars without gaining too much performance.
If you really have to upgrade, I agree with others, focus on the wheels and anything on it first.
Pinyo