I have found no perfect solution to this problem; only the mitigation resulting from vigilance, planning and communication.

The last time I did a rally (I don't ride many) the long courses and the short / fun-ride courses used the same route home, so the racer types doing the long course were cover the last 10 miles of rolling hills, mixed in with the casual riders on hybrids. My solution was to yell, loudly and often, things like "big group behind you!" and "riders passing"

My efforts didn't really cause any of the slow riders to move out of the way; they pedaled along in their zone of obliviousness. But at least I felt better than I had upheld the roadie code by yelling at them.