For me, really , the issue is all about precipitation. I am much, much colder after, say, two hours with constant rain and 43F than I am in dry conditions at 36F. So, my dressing algorithm, is first rain/no rain and then, temp/wind. I won't ride in ice or snow.

I stopped wearing a jacket a couple of years ago. I have an expensive one from REI that allegedly breathes, but I always end up hot and I hate being hot. Now, I wear a Proviz vest, which offers some insulation, and fleece kit +/- base layer, depending. I tend to be comfortable pretty much always now. But it almost never gets into the low 20s here, so keep that in mind.

To me, out here in the sort of rain capital, the whole idea of "staying dry" on a three or four-hour ride is just kind of a joke. There will be puddle after puddle and water and dirt everywhere. And then, you'll wake up and do it all over again. Why fight it?