Everyone has their own limits so it's hard for me to say what is too much for you but in my case, I have to have rest days if I am riding close to my limits. Like 2 or 3 rest days per week. I'm 66 and have been riding since the early 80s. Never raced, but have done 100 fast paced club rides per year for 20 years straight before this year.

Are you planning to race? Or do you want to achieve a certain level of fitness?

There are a number of forum members who follow strict training plans for racing and some of them are in their 60s and above. There is a racing forum here called "The 33". They will also be happy to answer your questions.