While it is true that as a percentage, the weight and speed differentials are small, you can definitely feel the difference. Your body adjusts to the weight that you are and doesn't perform all these calculations to figure out that the differences are minor. Try carrying a couple pounds with you all day and convince yourself you don't feel different. If you are a fit 140lb person, try strapping an extra 60 lbs to you all day and compare how you get around to people who barely exercise.

When I commute (particularly in the winter), I carry a lot of gear and I'm not a weight weenie by BF standards. However, I can definitely feel the difference when I lug more clothing or stop by the store to pick up a gallon of milk.

While it is true that shaving 3 pounds off the bike will only make you marginally faster, you can still feel it -- particularly when accelerating or on extended climbs.