Nope. As long as the insulation is not contaminated with sweat everything that is inside the insulation stay warm or loose heat more or less rapidly depending on your level of activity. (You can add a layer of aluminum fold to prevent even more heat from going out).
On the contrary, at extreme cold temp, with the classical breathable layers technique, the sweat which reach the outermost layer start freezing. After a while you end up with pounds and pounds of ice on the outermost layer.

But the vapor barrier technique as few drawbacks this is why it should be used only on hands and feet.
A few of them is that a proper VB decreases the range of comfort that's why this technique is good only for cold temp below 10F.
Another drawback is that you must wear as little layers as necessary with the VB technique because the VB increases sweat when you're overheating (for instance if you have too much clothes on you vs the effort) and your body which is unable to regulate heat by sweating produce more and more extrinsic sweat (extrinsic sweat: the one to regulate body temp, intrinsic: the one to hydrate the skin)

But if you use the VB technique locally (hands and feet), it's OK because you regulate heat from where it's breathable and you keep the most sensitive area warm (hands and feet)

One advantage of the VB technique is that it's independent of the length of your commute. Because the insulation layer is not contaminated with sweat, it keeps the insulation capacity the same whatever the length of your trip. (But you're getting hotter with time)