Before you quit the group, find out if there are others within the group that would appreciate a ride at your pace. Then all start together, but quickly divide into two or more groups based on speed. Since "No Drop Ride" is more important to you than speed, make the "No Drop Group" your ride, without moaning, complaining, bitterness, blame, excuses...
Just remember, There are lots of hot chicks in the slow group. Show them your leadership skills and diplomacy. Who knows. There may be a "littleGaryin LA" one day.