I ride my bike when it's 18 degrees ( F ) out, which is as cold as it's been in a few years in my neighborhood. I go kayaking when it snows, too; it's a pretty spectacular thing to sit in a small, low boat and watch the snow flakes land in the water all around you. Better yet, I wear mostly the same clothes whether I'm doing either of these, or hiking above timberline in the mountains.

Get a soft merino (wool) base layer, which means long underwear and a long sleeve tee shirt. Wear these against your skin, with wool hiking socks and glove liners, plus a hat. Then, add layers as appropriate. If you won't ride below 40 degrees ( F ) you won't need a mid layer, just a shell. It might need to be waterproof or windproof, depending on the circumstances.