Leather won't heat to your body as fast as synthetics, but assuming you're dressed for sub-arctic conditions, you probably don't care about that. I've never heard of a Brooks failing in that environment, nor, for that matter, any other saddle. Riding them soaking wet from rain has killed more than a few Brooks , though.
Remember that lots of stuff goes wrong at -20F, so what do I really know? Personally, I'd just drive or take a bus.