I've been filling one of those insulated bottles with HOT Chocolate.
Most of the time it isn't below freezing when I'm out, but it takes a while for it to cool down.
How long are your rides?
I also notice that while water is still nice, I can go quite a bit further without hydration than I was doing in the summer.