In my group when I lead out I leave them behind while not trying. In fact I try to go slower but look back sometimes and they are not in sight. Now this is usually on climbs. But like I said.......not a strong climber I have just gotten better. I have worked hard this year on my overall performance rather than one area. I hate long boring flats..........I like rollers; mountain climbs; and some tricky switchback descents. I think on flats for extended periods of time I would actually start performing less than my ability over a variety of terrain.
Keep riding and unless your a pro rider which I assume from your post your not don't get too caught up in this whole thing. Do what you can do and never be afraid to push it like one poster stated you either puke or feel like it. This is how you push that threshold to get better.