I'm a few years older than the OP (1946) and put in a lot of annual miles (between 9,000 and 11,000) a year. Up until 2006 (at 60), I only had a GP which I saw once a year for a physical. That year, I was diagnosed with a chronic type of leukemia which now has me seeing an array of doctors for other parts of my body. Never had or even seen a cardiologist until last year and he's only treating me for my blood pressure. Every doc that I have has told me the same thing and that is not to stop riding. I too, have thought about other types of exercise in conjunction with cycling but their answers were almost the same.Their concern was more with the cardiovascular aspect that with anything else. Since we're all different and have different medical and physical issues, I can only speak for myself as to where I'm directing my fitness efforts. However, I think that at the OP's age, if my doctor told me that riding was working for me, I would definitely take my doctor's advise and stick with what works.
__________________
HCFR Cycling Team
Ride Safe ... Ride Hard ... Ride Daily
2017 Colnago C-RS
2012 Colnago Ace
2010 Giant Cypress