In cold weather I use merino wool as the base layer. Add wind blocking/insulating layers as needed or remove as needed. I do carry a down vest to add as a quick body core protection layer when I stop riding. If your hands and feet are cold, make sure your head is covered! I am amazed to hear people complain about how cold they are and they have NOTHING on their heads.

Aaron