Overheating and then sweating as a result is what you want to avoid. I live in a hilly area and I find that in the winter, I have to be very careful to plan my routes and vent off excess heat while climbing so that I don't freeze on the descents.

I learned one winter when I XC skied every chance I got, that your body does harden off if you make a habit of exposing it to cold. It's amazing what you can stand if you're used to it. You probably even notice that when transitioning from summer to fall to winter riding. Hard to imagine riding in 40-degree temps when it's in the 80s in August. Once you get deep into the winter, a 32-degree ride seems like mild conditions.