I find that each year it gets easier. More miles in the legs means more muscle fibers recruited.
I would say that the time needed to recover increases a lot.

I can no longer play hockey 3 nights a week, play softball or soccer one day a week and cycle 6 days a week and have the success I used to have... and that's at 45.

But when allowed to recover, it doesn't get harder. You should be smarter and know your body better by then.

What is happening with the other 20-21 hours off the bike? Then I can comment on what is happening on the 3-4 hours on the bike.