I've found that around -10 C that these bottles: Camelbak Podium Big Chill Cycling Water Bottle 750ml/25oz - Mountain Equipment Co-op. Free Shipping Available will keep room-temperature water drinkable for about 2 hours before it freezes. The advantage of those particular bottles is that when the valve freezes up, it freezes *open*, not shut. If I need longer/colder I'll start heating my water before I go.

If you're looking to actually have a hot beverage, I'd follow GravelMN's advice of packing an actual thermos - just a little different when drinking on the bike.