Riding a mountain bike on single track helps. The lower speed equals less wind chill. Down to the teens I don't find I need much in the way of thick layers, just a warmish jersey, but do need to have a wind proof shell on top. Tights that are windproof in the front are nice when it gets to 20 and below. I wear my regular shoes with some mesh and medium weight socks. regular full finger MTB gloves are barely okay down to the upper teens. XC ski gloves are luxuirous, but my old ones wore out and it isn't worth buying new ones since I moved to Tallahassee. I had a pair for decades that I won from the prize table at a West Virginia MTB race and they were a prize posession until they eventually wore out. They'd have lasted forever here in Tallahasse

All of that assumes trail riding. On the road I'd add a layer on top, definitely wear warm gloves. and wear shoe covers in the colder temps. If it is really bad pogies are great, but I get off the road when it is really cold and have never needed them at trail speeds