I commuted for a week with biking shorts, and I rode to work today with just cargo shorts and a t-shirt (my commute's not long enough to get me soaked with sweat). I haven't noticed a difference yet. Maybe it's the helmet and lights.
Maybe if I ride without lights or a helmet, and ride on the wrong side of the road, I'd get the same respect that the other salmon get. Heh.