People generally suggest upgrading wheels because A.) a lot of bikes come with soft, heavy wheels, and B.) it is a much more meaningful upgrade for most bikes than the ones people do.
If you can save a pound of rotating weight you are improving your bike's performance potential a lot more than the almost imperceptible change of going up one level in group set, and way more than a new stem or seat post with a flashier logo.
Strong, stiff, light wheel waste less energy. Every time you speed up and slow down they offer a small benefit.
Will you be able to feel that benefit? Maybe doubtful, maybe imaginary ....
I swap between a CF climbing bike (not super light but pretty light) and a pair of aluminum endurance/tourers which weight 25 or 26 lbs ... 8-10 lbs more.
When I first climb onto the different bike I can tell ... for at least the first 20 yards.
Then it is just another bike ride.