As stapfam said, beyond the first few months (although for me it was the first couple of years) you can no longer count on "automatic" speed and strength gains from just riding around. Now you have to work for it.

I'm 54, in my sixth season, I work at it, and continue to experience measurable gains in strength and speed. Even so, it comes slowly over time. Perhaps if I used a regimented training program it would move along a little faster, but regimentation and my personality don't make a good fit.