Sorry to hear about your accident! My teammate broke her pelvis in two places earlier this year, and it healed pretty quickly. She was working out in about 6 weeks, and racing within 8 or so. Her fitness came back very quickly, but the confidence was a little slower going.
+1 to listening to your doctor.