I would argue that the fork is the most significant upgrade. Upgrading to a good fork verses some of the cheap forks on most entry level bikes will make a HUGE difference.
From there I agree with those who say, "WHEELS" or rotational mass, meaning, tires, tubes, rims...etc.
The 1 to 3 means that if you reduce your rotational mass, it will "feel" like you've reduced your static weight. Your bike will accelerate quicker, handle faster and feel livelier. This is due in part to the gyroscopic effect your wheels have on your bike.
Getting new wheels is expensive, but you can easily shave .5 lb off your bike just by going to a lighterweight tire.
__________________
"Life is not a journey to the grave with the intention of arriving safely in a pretty and well preserved body, but rather to skid in broadside, thoroughly used up, totally worn out, and loudly proclaiming, "WOW, What a Ride!" - unknown
"Your Bike Sucks" - Sky Yaeger