Something is clearly letting freezing air inside. As many pointed out, balancing front wind protection with breathability is a common problem with cycling in cold weather and it works differently for everyone. A vest or jacket designed for cold weather will be often referred to as a "barrier" vest or jacket. The entire front is windproof and waterproof (including a good zipper!) and the sides and back are breathable. Not every cycling vest or jacket is designed like that.

I have noticed that the areas on me that are most padded also turn blotchy red/purple after I finish my ride. I don't notice it while I'm still riding. It can happen to me with temps below 48F.

I assume it is because the fat insulates the skin from the warmer core; and the surface blood vessels constrict to keep the core warmer.

It is itchy but I don't think it causes any problems if it's back to normal within an hour.

Thanks for making this post - none of the other people I ride with have this happen to them, so I've just tried to figure it out on my own.

Give it up -- were you trying to ride shirtless? ;) No, I have a "non-traditional" opinion of over-priced bike-specific clothing and am not the least surprised that a thin plasticky vest didn't work for you. Others have mentioned trying wool, but my personal choice is regular LS shirts and a good polartec (polarfleece) vest. Those really cut the wind and work great for me -- easy to move in too.

Gotta keep your core warm! You're not a shaved head guy like me, so you're probably good to go there ;).