My commute doubles as my cardiovascular exercise. I will sweat like crazy so I don't wear my office attire while riding. It's the same with the runners/joggers at work. They don't go for their lunch hour run in their suit.

Now, if I'm running a quick errand on the weekend (like a couple miles) then I'll ride in whatever I'm wearing.