I rarely break a sweat when alpine skiing... cycling is just so much more aerobic.
I've yet to feel a need to wear my ski helmet while cycling. I think it'd have to be below 0°F. Granted, I have a helmet cover, so... plus, it's got reflective strips on it, so that's cool.