I've had the same trouble before. It seems to be wind related. If I wear only something insulating--like a fleece--the wind still gets through and turns me red. I need to wear something wind-blocking and then, less insulation to compensate.
Also, my HRM strap made it worse. The sweat would run down from under it, making my belly wet and colder. I no longer use my HRM in winter.