I used to have a really short commute. As a gift I got a really nice pair of light weight long underwear (base layer) for skiing and running that breathed well. I ended up using them for commuting too. Having never worn long underwear all day at work I figured I'd take them off when I got there but it turns out they didn't make me overheat at all.

Under Armor sells good base layers in 3 different weights. I know you said that you didn't want to wear long underwear at work but you might be pleasantly surprised. It was nice to have my lower half all set to go when it was time to go home.