I do think its part of the cyclist rite-of-passage, thinking that everyone on the road is out there to race, and race you specifically.
Moreover, after a whole day of riding why would I need to keep up with someone who might be "just put on my spandex" fresh.
When I used to do it, it was less about the cost of the bike and more about the fact that I'm riding an upright mountain bike dusting you on the flats with knobbies.
Now its the other way around, I enjoy if people want to do this to me. I ride up to them after they (try) to dust me and see if they're really game, had a nice competitive run between Corvallis and Albany the other day.
Like talking to yourself, racing without the other person knowing is not right.