I wear shorts at work through most of the year as the winters are mild in NZ, check out your Surf/skate clothing shops, there are usually plain black shorts somewhere and some are quite smart looking. I also don't wear my cycle shorts at work as they can get wet when raining or sweaty when it's warm. So I've currently got a nice pair of black quicksilver shorts which I put on after my shower.