I am curious if this group was a club or a shop ride. Often shop rides can be rather fast paced. My local LBS has shop rides such as this:
Its a 20 mile ride to the base of the canyon. Where we will climb up Turnbull Canyon for the 3.5 mile climb of switchback and then head back to the store. The total mileage will be about 48 miles with an average speed of 22 to 25 mph.
I am not anywhere near being able to ride that fast!!! So I don't go. At least they are telling me they plan on riding like bats out of hell. 
I recently left the women's club I joined because there were too many "rules" as to how fast we could go. We were a no drop deal and that is fine but I figured why not just set up regroup points and make sure we have a sweep if there are some who are stronger than others so as not to force those who are stronger to go so slow as to not get any benefit fitness improvement wise out of the ride. I rode with another group that was set up that way....they had a designated sweep and there were regroup pit stops set up. Other than that you were welcome to ride at your own pace. I tried at one point to keep up with the faster riders and got dropped. I had my route slip and eventually met up with them at the next regroup point.
I have been told that I am a stronger rider than I give myself credit for and that I might need to start pushing myself out of my comfort zone on my rides.
Another new group I am riding with (less formal than the club) is really good about detailing the type of ride and what the expected pace will be and there are regroup spots but there is no "stay in one pack" rule. Coming up is a "tempo ride" where we are going to push the pace up (aiming for a 17 mph/avg). Looking forward to getting pushed out of my comfort zone.