I'm 45 and have thinking about this sort of stuff lately too, but not in regards to cycling. I know a guy in his 70's with really bad arthritis who commutes. A few years ago he rode from the midwest to the East coast with his daughter. He's had both a knee and a hip replaced.
I have a neighbor in his 60's who also commutes and regularly does long distance rides including some 50 mile races. In the triathlons I participate in there are always guys in their 50's in the top 10% of finishers. I guy around 50 won one of the smaller races I was in a few years ago.
In my case I was concerned about soccer, which I recently started playing again after about a 3 year break. I play with guys in their 20's and 30's and was worried about embarrassing myself.
What I have found is that fitness is not something you lose by aging. You lose it by sitting on your butt. I'm not kidding myself. I'm not as fast or quick as I was 20 years ago but I'm not as slow as I would be if I were carrying 20 extra pounds like some of those kids I play against are.