Moving is the best way of maintaining body temperature in cold weather. That's why you start to shiver when you're cold enough, muscle movement warms up your body. Being surrounded by cold air doesn't make your muscles slower or weaker per se. Elite XC skiers cover 50km of very demanding competition track in about 2 hours, in temperatures anywhere down to -15C. I know I would be hard pressed to ride the same track in summer in that time. The temperature limits in international competitions (-15C means precautions, -25C means delayed or cancelled competition) are to avoid frostbites on exposed skin (face) and effect of cold air in lungs. And those limits are for professional athletes, in a sustained exercise at the top of their performance. Their muscles seem to be doing fine.

In conditions where I ride, cold air is more of a psychological factor, if even that. In my experience, biggest slowing factors for me as a commuter and utility cyclist are rolling resistance (caused by studded tyres, lower PSI and snowy/soft surface), extra weight and layers on self. In very cold weather, drivetrain and hub lubricants, although those tend to warm up somewhat during riding.

--J