i ride for my commute 2-3 x per week. I figure if I have to commute I might as well get some exercise out of it. When I start to feel like my fitness is leveling off I'll start adding miles to the morning ride. In the winter I use studded tires, and with the snow/ice it's hard enough that I don't add miles. I don't worry about how fast because I'm not competing against anyone but my own fitness goals.
Sam