I'm one of the weird ones who likes riding in the cold. I would rather ride in cold weather like we're having now (lows in the teens and 20s F) than hot humid weather we get in the summer with 95+ highs. As others said, acclimation is important and you have to keep riding when it starts cooling off in the autumn to be able to handle winter. You also need the right gear. Hands and feet are the weak links, so you need decent gloves and shoes/socks/coverings. Keeping your core warm will also help keep your extremities warm, but overdressing will make it worse because you sweat. It's a process of trial-and-error knowing what combinations of clothes to wear.

My biggest discovery the past few winters is the benefits of rain jackets for general purpose riding in winter. A good rain jacket will also function as a wind block but allow good ventilation. You can wear various base layers and jerseys under the rain jacket, depending on the cold. This past fall I bought a new Showers Pass Elite 2.1 jacket and it has been incredible for the cold weather -- extremely well ventilated yet great at blocking wind. As a bonus, it's got me covered if rain crops up.

BTW, I set a new record cold commute for me this morning, 12 F with a wind chill below zero. However, I'm don't think I can handle temperatures much colder without investing in some warmer gloves and shoe covers. That probably wouldn't be worth the cost because we rarely get temps below 20 F in this part of NC. The weather we've had lately is the coldest here in more than 10 years.