To give you an idea of the numbers here, OP, let me tell you this:

The difference between a $2000 wheel and a basic but good wheel is about a second per kilometer, maybe slightly more.

If you are Fabian Cancellara ... those few seconds per kilometer will after 40k ... add up to about a minute of lost time.
A minute, in the world of time trial races, could mean the difference between being first and not even being on the podium at all.

If you are Average Joe ... or even Wannabe Fred ... those few seconds per kilometer mean next to nothing at all.

I hope that cleared it up for you.