Sorry to hear of your bad experiences. In my own personal experience I've come to understand that if I think someone is riding in an unsafe manner, I have a responsibility to protect myself (after all no one has more of a responsibility to do this than me). Hence, I probably would have slowed down and indicated that he should ride in front of me. If we wasn't willing to do this, I think I would have pulled over to the side of the road and had the "argument" you worked to avoid. (Although I suspect it wouldn't be much of an argument if I simply indicated that I was not going to continue to ride in the manner I deemed unsafe.) I don't think group riding in and of itself is bad. I think there are some bad riders out there.