I think whenever you have windchills in the -20's (Celsius), it's too cold, though nowadays I don't do any more extra miles before getting to work, and so being out for 50-60 minutes in -20 might be tolerable.