Living in the flat lands (so we still have hills, but they're not very difficult) I do. Wind is the key determinant for me. If it's still, I'm gonna be strong. If it's 20mph winds, I'm gonna feel weak.

According to icebike your aerodynamic resistance increases (yes, noticeably) as the temperature decreases. So 80 versus 60, you might not care. 80 versus 20, yea that's a difference.
http://www.icebike.org/Articles/SlowerWinter.htm

There's also your clothes and the fact that your body doesn't want to work as hard.