There can be the attitude of 'I own one full lane' but unless it is necessary I tend to stay towards the side. I look at it this way; if there is a wide shoulder, bike lane, or extra wide roadway then I am being presented with an opportunity to be courteous to another vehicle operator. I am generally a nice person and I stay as far to the right as is comfortable.

As far as group rides, if you were next to a guy and were not passing and the road would have allowed a car to pass if the cyclists were single file, then when there is a car you get over... If there is not a car, do whatever the heck you want. I always want a cyclist to be safe, but safe does not mean only use a bike path and it does not mean staying in the middle of the right most lane that considers your directional and speed positioning... it is somewhere between those options and you should use your brain.

Next time the guy speaks up, tell him that you respond better if he talks to you and tells you not only what you are doing wrong, but also why he believes it to be a violation of club ettiquite (sp?) so that you are not a disruptive influence.
or tell him that being yelled at makes you froth at the mouth and want to shove his pump into his own spokes...
or tell him that if he spent a little less time worrying about other people's riding style then he would have more energy left to pedal faster...