It's sad when it is 20F and my first thought is "it's warm!". Of course, those further up north will find that funny.

I tend to overdress. Temps vary throughout the 30+ miles I ride.
Take one layer off. 45 minutes later put it back on.