To go back to the OP's question, I don't think there are a whole bunch of people out there that behave like that.

I use my commute (both ways) as exercise, and I do ride fast the whole way. This means that I will pass a few people each way. Mostly, they do not react, but occasionally they will speed up and try to pass me. This does not bother me in general, as long as they keep up the pace, and don't force me to pass them again (even then I would not mind if I didn't have to keep passing and re-passing; but that's never happened to me). Since I stop for all red lights and stop signs, this gives the time-triallers ample time to pass me (I've noticed that someone who so desperately wants to pass you will generally run reds).

Still, I guess if somewone was ahead of me and I passed them, they'd think I was racing them, when I was really just doing my thing.

I had one somewhat funny experience on a commute, which is probably quite common: I was riding in to work, doing my usual pace (19-21 mph, unless there's a headwind), and I was on this nice, straight road with a decent bike lane. Up ahead of me I noticed another rider. The road was slightly uphill over a 2 mile stretch, so I wound up catching up quickly, and noticed this generously proportioned guy on a nice bike with the saddle too low ambling on at a nice easy pace. Anyway, I passed him an went on my merry way. About half a mile further on, I had to take a left and had to merge through two lanes of 40 mph traffic, so I started looking back at regular intervals to see when I should signal and start the merge. I see this guy about 20 metres behind me puffing away, and really straining from side-to-side, looking like he was going to have a coronary. He actually did quite a good job of keeping up, but I hope he did not overreach himself. He could have been doing intervals, I guess.