Last weekend I went on a nice casual ride. Very... Grant Petersen style. Although I did wear a helmet... but I used no other cycling specific clothing. I even rode with platform pedals.

I have no idea what percent of cycling clothing is style and fashion... and which part is useful cycling equipment. I think for the most part most people could bicycle very happily with no more equipment... than the bicycle itself.