One of my pet peeves with group rides is that the guys at the front often think like they do when riding solo, rather than like they're driving a really long vehicle with a really long trailer.

In this case it sounds like a typical thing, where the front guys waited to move left to pass the slower cyclists until about the point where they would have moved left had they been riding solo. That doesn't work with a large group. You need to look back to make sure it's clear and merge left early enough to allow the entire group to safely establish right of way further left well before you reach the slower cyclists.