Might if you don't cover up.
As stated in OP, a 'perfect storm' type scenario where the extreme low temperature combined with high winds and dicey road conditions makes it a no-go day for me. The only factor that is somewhat tolerable, meaning safe to ride, is the low temperature, but it is not without its challenges. For me it means trying to keep my fingers and toes from getting numb on the latter half of the journey. Chemical warmers are helpful but it's far from perfect.