I bring an extra layer with me when it's cold in case I end up not being able to ride, like when I got a flat tire last ride home of the year for me a few years ago. It was dark and rainy then, and I made the call of shame instead of changing the tube there. The extra layer kept me warm while waiting.
I bought a new winter coat this year, and was amused at reading the reviews. One was from someone in southern California who said it kept him warm when the temperature gets down into the 50's, and another reviewer said he could wear either layer until it got down into the 20's and then combined them for more warmth. Those 2 definitely have different definitions of cold.