Depends on what's broke. I was off for almost 6 months for a tiny bone in the wrist that doesn't heal well. I am still getting back to my post-accident capacity, but it is going a lot faster than I thought. I would say I'm about 85 % after 2 weeks of slowly increasing my time and speed.
You might ask your doctor if you can ride with the cast. My doctor said I could (but I didn't, for unrelated reasons). His exact words: "Riding a bike never hurt anybody. It's falling off that hurts. And if you do fall, try to fall on the other side so you don't mess up all my work." What a comedian!