The guys I used to ride with regularly (mtb and road) used to ask me this all the time... a friend who races at a Cat 3 level said I would have destroyed most of the guys he raced against and was thankful I was twice his age and would have to enter at a master's level...
I never gave any thought to racing aside from friendly little competitions with my friends and half a lifetime ago might have even been able to compete at the highest levels... people who think I am intense now have no idea how much I put into riding then and that was just for my own satisfaction.
I was actually considering racing a few years ago to see what I was still capable of and was training harder than I had in decades... I fragged my back so badly at work that this is now impossible but I'm still happy that I can still get out and ride everyday and most of the time enjoy a more leisurely pace.
I still have moments when I can dial it up but the body can't take that kind of abuse and I pay for it afterwards... and I can't afford to be laid up like that as I have other responsibilities.