I was going to say my morning commutes are slow because I'm a little tired and I like to avoid getting to work all worn out. Then I looked at my spreadsheet (I'm a geek, feel free to judge if you must), I found my mornings are slightly faster this year - I was surprised.
For the record, I averaged only rides that were on the same route using the same time standard (moving time for my spreadsheet). I noticed I like to use the 'direct' route more in the morning than afternoon (2014: 49 vs 22), I run errands on the way home from work and mix up my routes for fun.
My route is North/South, so winds are equally varied. If your commute is East/West you could see a consistent wind-effect on your commutes, assuming dominate winds out of the West.
The only way to get a good feel for your commute times is to look at statistics over a long period of time over the exact same route. By using a larger sample, you minimize the wind, lights, traffic, and other variable issues.