I typically hate it when it first gets cold in the Fall. Granted, I'm in the southern U.S., so "cold" is relative. But as the winter progresses, I do get some satisfaction out of riding in the 20's and 30's. Makes you realize you really want to be out there. I'm a nearly-everyday rider, mostly in the morning hours, so I get a good bit of this in the winter. Good winter riding gear is a must and so worth the investment IMO.
The last mountain bike race I did, in late November, had temps in the mid twenties. For a bunch of southerners, it was as much a battle of survival as it was a race. I did pretty well, but more than that, it was satisfying because I was one of the ones who showed up in tough (for us) conditions. We'll talk about that race for years, I'm sure. Big bonfire afterwards, too.