I think heat is worse, because your body can only cool down so much, even naked if the ambient temp is over 100 you'd need a pretty strong breeze just to keep out of the danger zone of heat stroke.
I've ridden in 100+ temps, and it is worse. Below zero is way better than over 100.
- Andy