Just a note from a young guy...
When I'm riding with a group, and especially when I'm the one setting the pace, I try to keep the speed controlled so that everyone in the group is able to comfortably keep up, including the older riders. The people I ride with regularly have even recognized this and have named me their "official morning ride puller" for a short portion of the ride on the way out and back in.
Dropping cyclists twice or even three times my age (we had one guy join us that was 82) just isn't where I get my kicks.