If I can input my 2 cents, it depends where the weight is coming from. If you're taking 3 pounds out of the wheels (aka unsprung mass), you'll notice the difference in how the bike feels and performs. It'll have more of a racy/rocket ship feel to it.

If the weight is coming off the frames or components (aka sprung mass), you're not going to notice much of a difference at all. People tend to get too caught up in a few grams here or there that make close to a negligible difference, particularly on flats. Avoid getting sucked into the law of diminishing returns by some weight weenie and save yourself a few grand. If anything, consider it motivation to lose a few pounds off those love handles.

In the same vein, I've become a fan of steel frames for the rigid, racy feel, despite the additional poundage.