Rebel against the man and wear whatever you would like.
Seriously, I keep one days worth of clothes in my desk. Since I do not sweat at work I use pants for 3 days and bring in a shirt and underwear each day. My work shoes stay at my desk.