When it comes to things like endurance, pain tolerance, sustained pedaling speed and cadence, there is a real need to get out of your comfort zone; you'll never improve until you do.
When it comes to things like downhill speed, cornering speed, technical skills handling, extreme terrain...out of your comfort zone becomes a 50/50 chance of disaster. THOSE are the times when you ride TO THE EDGE of your comfort zone.