Oh, I have had the same thoughts. It seems that crashing is inevitable, and as we get older, we are more fragile and recovery takes a little bit longer.

I don't even know how my last crash happened. I woke up in someone's car, and when he noted I asked him the same questions multiple times, thought better of bringing me home, and brought me to the hospital instead. The damage:

2 broken arms: (just think about the things you cannot do when both arms are in a cast).
Broken occipetal condyle: That is a kidney bean shaped bone that is at the base of the skull where that rests upon your Atlas ... the uppermost bone in your neck. This kind of injury typically only happens in serious car accidents involving head on collisions. I was lucky it was not displaced, but that still meant wearing a neck brace for 4 weeks or so. If it were displaced, I would have expired.
Broken nose: No loss here, my schnoz was never much to look at anyway. But in the "minor procedure" they had setting this, they scratched the hell out of my corneas, and let me tell you how uncomfortable that was. OMG.

That doesn't even count the matching broken collarbones and multiple broken ribs I have from previous crashes. Oh, and of course, pesky road rash.

I know multiple people who are no longer riding because of crashes. A professional trainer who is now using a walker (and frankly, lucky to be using a walker at all). Another friend who got such substantial brain damage from a crash, he is a shadow of his former self (yes, he was wearing a helmet). Another struck by a car and killed. Another having a massive MI, and right after getting a docs OK to ride.

But I can't imagine giving it up. I love it too much. All I can do is take reasonable safety measures (riding on quiet secondary roads as much as possible, bright clothes, helmet, lights, keeping my speed down and not taking excessive risks). And I do that knowing that ultimately, it is a gamble.

I ski as well, and that also has it's risks. I have friends who think I am nuts to do any of this at "my age."

I don't think there is an answer, really. But fish gotta swim, birds gotta fly, and I gotta ride. If I don't, I'll die a slow diminishing death instead of risking a spectacular one. Which is worse? I guess we all get to decide for ourselves.