Some things are huge in comparison to wheelset weight. Ride a MTB on pavement with the tires it came with and then put on 1.25 slicks -- yes you WILL feel the difference. Other examples in no particular order are: reasonable fit, properly working shifters, appropriate gear range, ditch that suspension seat post, lock out suspension fork or replace with non-suspension (for riding on good surfaces.) Once the bike fits you, shifts reliably, has a suitable gear range for your circumstances and has nothing actually wrong with it, returns diminish rapidly as the cost of upgrades increases rapidly.

The more you spend, the less bang/buck but there will always be some. You could spend $5000 on a $3000 bike but don't expect a quantum leap in performance.

Don in Austin