I don't race, and on some days I wear cycling shorts and jersey and other days I wear cargo shorts and a t-shirt or cinched up pants and a tank top. My clothing choice depends on my daily agenda and the forecast.
Unlike Mr. Capolover, I don't have the mutant superpower that prevents me from sweating on a 6.5 mile commute. My 'normal' clothes do get sticky and stinky from my ride and I always have to take a shower regardless of the kit that I used to ride in. I do tend to prefer the plastic jersey and shorts because it stays cool in warm temperatures and it dries quickly in wet conditions ( I never wear anything with cotton or denim if the forecast calls for rain ).
However, I'll wear street clothes if I have to do a lot of intermediate stops between home and work (ie. grocery, bar, movie, etc.); and I'd rather have the keys to my lock in a hip pocket rather than jersey pocket. Ironically, the peer pressure involved in not looking like a "gay superhero racer weenie" plays a bigger role in my decision to wear street clothes. I feel absolutely no pressure in the other direction. It just seems that everyone likes to make fun of the roadie outfit despite the fact that it really has some useful functionality.
As to the original question -- on my commutes through Cambridge, I normally see a ton of folks riding in street clothes and maybe only 10% of the riders have a cycling jersey and/or shorts. I see a rider in full team kit once every three days.