yeah, don't worry about their age. You'll realize that age only matters to youngsters like you who have only been around people their own age. I'm 33 and tend to be the youngest on some group rides. Does this matter? Not especially unless I only care about what the occasional driver or pedestrian may think for .003 of a second.

You'll come to realize a few things:

Endurance sports favor the older.
These "old timers" may have already done many of the things you aspire to do, learn from them.
Many have been riding long before you were off of training wheels.
That "slow" 17mph pace you hate can continue all day, long after you bonk.
That "slow" 17mph pace may also continue up the hill, and up the next, and the next.
You never know when they are "just taking it easy" that day because yesterday was a century or an interval day.

And lastly, as you mature you'll realize people are people and you'll learn more, learn faster, broaden your horizons and become wiser if you are open, friendly, ask questions and listen more than you talk.

Good luck and have fun! If you need to do a lung-busting-mountain-interval the day before then so be it.