Unless you're successful at one specific sport, you should be looking at other activities to round out your fitness regimen so short spurts of time off from one activity will not severely affect that activity.
Run, lift, swim, Cross Fit, etc. etc. in addition to riding to stay in good overall shape.