Why the concern about speed? If you want to go fast, get a motorcycle!
Most of the 50+ year old riders I see are going your speed or slower.
It all depends on what you are trying to get out of your cycling. It sounds like you might be looking for more. If you are looking to be race competitive, then you will have to train yourself up to a faster speed. I'm a bit faster than most in the 50+ group (average 20 mph for my 50 km training ride, 18.5 mph for the 100 km training ride, and 17 mph for a "ride all day" pace). In my 20's and 30's I rode LOTS of miles (5,000 to 6,000 per year) and in my 40's I cut way back on my riding but did other things to stay in reasonable shape, so I'm not coming into this "fresh". Back then I wasn't racing and rode at speeds similar to what I ride now. I don't consider this to be anywhere near bike racer performance, but I find it enjoyable and I get a great workout. I do use cycling for cross training for fencing which I do competitively.
All riding is good. Last weekend I rode most of my 30 mile training route with my 9 year old son with me on my commuter bike at around a 11 mph pace. While I didn't get that much training benefit from it, I got a lot of enjoyment from it. I never drop my son and I hope when he's 18 (and I'm 60) he'll return the favor.
