Of course it matters.
The question really is:
"How much does it matter to you?" or even, "How much is it worth to you?"
Let me illustrate. Would a bike weight of 75 lbs. be a lot? Could we tell the difference between that and a 17 lb. bike? Of course we could. Weight does matter. Can we feel the weight differences? In this extreme example, yes. But if the difference is only 1 or 2 lbs, this might be too small to actually feel. That doesn't mean it's not there. It is there and it does actually require us to put out more effort--even if we can't tell. It all comes down to degrees... er... pounds.
That being said, there is another factor to consider:
How much weight-reduction is too much? At which point is the lighter material/bike too fragile?The best example for us clydes is wheels. But there are others. When the LBS sold me my bike, they asked if I wanted a carbon handlebar & stem. I said, "No." I felt that the reduction in durability/reliability of the bar was not worth the extra expense ($$$). Plus, I figured, if I wanted to reduce the bike system's weight--rider, machine, and everything we carry with it--by another 4 ozs., I could just go to the bathroom before riding.
And perhaps the final follow-up question is:
"Is the $2,000 you'd pay for a minor weight reduction worth it to you?"Well, if you have the money, and it makes no difference to you, go ahead. Have fun. It's your money.