Think of all the cold air you're sucking down. Your body has to devote energy to staying warm.

Try a breath warming face mask. I use a Psolar.

On the plus side for some, I suppose, is the fact that a lesser degree of intensity can produce more calorie burn.