I've had some very pleasant rides in the Winter but I've also had some of my longest and most nerve racking commutes when the roads are bad.
The plus to both Summer and Winter is that I can wear the same clothes both into work and on the way home. In the Spring and the Fall it often warms up enough during the day that I end up hauling half my clothes home instead of wearing them.