I reckon the main issue might be grease/lube thickening which can lead to drivetrain problems like stiff chain, freehub seizing up etc. Suspension could be affected too, and I have noticed an effect on mineral oil hydraulic brakes as well - they still work, but the lever feels different.

Regarding the toughness of materials, steel does become brittle at lower temperatures, but it varies a lot depending on carbon content and alloying elements such as nickel, chrome and others. Cheap hi-ten steel can become brittle at temps as high as -30°C, but other types of steel can lower that quite a lot. Aluminum doesn't become brittle in lower temps due to a different crystalline structure.

So you should be fine on aluminum bike, but jumps could destroy steel parts at extreme temps.