They make garment panniers that minimize how much you have to fold up your clothes. I haven't tried them since I try and wear golf shirts on biking days (allowable at my work) and these fit in my Nashbar ATB panniers. A dress shirt will get a little wrinkly in these I've found. You could try that anti-wrinkle spray but YMMV.

I bike in athletic wear (wicking) and then change at the office. I bike slower in the morning than the ride home. I definitely recommend the baby wipes as mentioned above. This plus clean clothes and deodorant makes all the difference. I also Febreeze my ride clothes since I wear them home and I don't want to stink up my office or the changing bathroom when I'm getting ready to come home.