I wear the same clothes home as I do when I ride in except for making any weather related adjustments.

As long as it's not cotton it'll usually be dry for the ride home even though I keep my stuff in a ventilated gym bag under my desk.

Note: I also use tri-shorts in the summer. They dry pretty quick.