A bunch of people ride in their work clothes here.
I'm pretty much a "change when I get to work" guy but that's because I use my commute as a workout. However, I've experimented with doing my commute in regular clothes and taking it easy to see how workable it is. I've planned on doing some longer posts about my results but basically if the temp is moderate, the route is fairly flat, and you take it easy enough, you don't have to work up a big sweat.
If you find that doesn't work for you, don't sweat it.
Honestly, too many people are far too concerned about how other people dress for their commute. Obviously not having to change makes things a lot simpler. Changing has it's advantages too. Choose which ever makes sense for your situation.