To the point of yelling "Pull Off," I'm not sure I can comment since I don't do group riding, but I can see why you might want to get back up to the pace. There are times I've gotten behind another cyclist going a bit slower than I like and I just dig in, accelerate, and say "on your left."
==========================================
RE: Those offended by sigs.
In the words of Sgt Hulka
"Lighten up Francis."