Like others have said..tell the group leader you do not mind being dropped. Tell him/her you want to be pushed and riding with a faster group pushes you train harder. If the group leader doesnt respect that, then its a group you probably do not want to be associated with anyways.

In just about any sport I have played in, I have always trained with guys faster and stronger than me. Training solo I just dont push myself as hard.