Yes, it's horses for courses...

Everyone has a different commute, a different location, and a different job (different dress code). There is no right or wrong way here.

I commute to work in work clothes this time of year, because it's cold and I won't sweat too much. For the ride home, however, where I like to push a little harder, I'll put on athletic clothes (not bike specific). But if I were planning to do some sort of serious ride before or after work, I wouldn't think it was strange to put on cycling clothes.