I'm 58 and have been riding for 30+ years, and I haven't really seen much slowdown yet.
My max pulse is down from 210 to 185 so I can't sustain a sprint as long, but my endurance is good and I regularly ride 1-2 hours at 20+mph, and the occasional century and double-century. I still set new PRs over times I set 5 years ago.
BUT I need more recovery time; I can train 3-4 days a week tops, and if I do a race or a hard ride I need 1-2 days of recovery.

I use a power meter and Trainerroad to structure my training which includes intervals and periodization with easy weeks after a couple of hard weeks.
I think with proper training (and recovery) you should be getting faster not slower, since you were probably not as well trained 5 years ago.

Why don't you share a typical week of your training/riding and see what kind of feedback that gets you.

Btw I also agree on with RandyO on the nutrition side. I generally avoid fast carbs and processed foods (after reading Grain Brain) and eat far more vegetables, berries, and nuts than I did 5 years ago which resulted in my dropping 5 pounds pretty quickly and am back at my college weight of 155lbs (5'10").