The key is to dress in layers. The first or "base" layer would be something like UnderArmour cold gear. There are lots of options here. It fits close to the skin and wicks sweat.

The next layer is the "warmth" layer. I use a fleece pullover.

The third layer is the outer layer and you want something that blocks the wind. It doesn't have to be thick. Typically it's not.

On my legs I generally don't wear three layers. I'm not sure what you regularly wear but in the 30s you might be OK with that or by putting on a base layer under it.

A skull cap is generally fine for me until it gets into the 20s or lower. Then I'll wear a balaclava. A company called Seirus has a nice variety.

Notes: For natural materials, it's best to avoid cotton but wool is wonderful.