I can relate...sort of. I did my first group ride last week. A 50 miler. Well I was late and it was just me and another rider. She was perhaps 25 or so years older than me. I thought, okay, I can keep up with this chick. I am fat, but I have been riding a lot, lost 20 lbs, and I am way younger than her.
Well, I was able to hang on for dear life for about ten miles. Then she gradually got farther and farther away. My legs just couldn't do it. Her cadence was REALLY slow, but still she just pulled away. Because we did the same route we kept meeting up at various logical stopping points. Since this was a long ride for me, I figured my best bet for finishing was to go at my pace. But it was really good to be pushed.
I would not have been pushed that hard if I rode alone. When I ran track as a sprinter, I was pretty fast and often the fastest girl on the team. So they had me run against the fastest boys.
The best way to get faster is to ride/run with those faster than you. It may boost your ego to continually beat those slower than you, but you will not improve like you will if you are the worst in the pack.