I have a hard time with my feet getting cold. I had to stop and use toe warmers. After moving them to the bottom of my foot at the coffee shop, I was toasty warm on the way home.

Here is an idea for winter rides - "coffee shop ride". We go out ride 10-15 miles to a coffee shop. Warm up and then ride 5-10 miles home. This is in 20-30 deg temps. The other approach is to ride some place for lunch - have done both this month. Last year I kayaked at least once a month all year round. This year, I am going to focus on year round biking

I have new REI jacket and wind pants for cycling - what a difference. I use ski gloves and my hands are OK.

My bad knee has arthritis but the knee joint loves cool dry weather - it keeps it from swelling.
What I don't like are weather changes and high humidity.
I assume I also have some mild degeneration in other joints from various past fractures dislocations overuse and/or lack of conditioning.
I'll be the one in summer attire on a sunny calm 55F morning.
I'll be wearing winter attire for a 55F winter dark night ride with bitter winds.

Most cold weather rides I'll keep the distance down and aim for a moderate effort to keep thermal regulation easy. Hard enough to generate heat, but not enough to sweat-soak (then chill) myself.
Below 40F the air is harder to breathe so I don't go fast anyhow.
Coffeeshop / breakfast rides are awesome on cold days.

Even my Lake boots don't keep my feet warm enough when it gets below zero. I ordered up some electric socks to see if I can keep them warmer. I found some where the batteries clip to the belt. I can't imagine why I'd want to spin the pedals or walk with a D battery between my ankle and knee. I'll write up a report of them after I try them out this weekend.