I often see people riding on my way home and ask myself the same question about why I didn't ride to work that day. The simple answer is not that it's too cold in the morning right now, but that it's too DARK. Driving to work every morning with my windows barely defrosted, I see how little attention drivers (including myself) pay to anything other than cars out there. This is the country, where there are very few bike commuters even in good weather, so nobody is looking out for bikes in January.
Yeah, I am jealous of people riding at 4pm, but no thanks...given that I have a choice about riding to work at 7am these days, I will wait until the mornings get a little brighter and the motorists have fewer distractions.