Hey you only cycle and swim (as far as I can tell). Try being an over 50 year old Rugby player! Every weekend I get banged up a bit, sometimes more than a bit. I finally got my mom resigned to the fact that I'll never stop, but still have to convince the wife.
Seriously, if you enjoy cycling then it is what you have to do. What is the worst that can happen? You die. But if you give up everything you love then you are dead already. And everything is dangerous in some way.
BTW the fitness and reactions you keep at topp level by cycling and other 'dangerous' physical activities have a pay off in giving yuo a better chance in those slip and fall in the bathroom accidents that kill many.