Personally i hate to say something, period. If there is room to the left to pass i just go. What surprise me the most is that people do not pay attention to a single thing while riding. I can hear guys coming from at least more than +20 feet away sometimes, it is so easy also just go and look over your shoulder also. I got the bad custom from my tracking days but anyhow, people just go and ride and use only the sight sense and tha's it.
The other bad custom is people just waving while riding the bikes, from side to side of the road, no matter what u say, that M O F O could cause an accident, and no matter how to explain to them, they will continue doing it. The other thing, if you say To your left of whatever, some get scared and dont know what to do, making the situation even worse.
What i have found useful is just wait, do not say a thing and wait for the right moment to pass. If you are passing and they start moving to the left just touch their hips and move them pretty slowly like for 2 secs, another thing i got from my racing days. Nobody is hurt u are sure the stupid guy will move and everybody happy. The other detail, women tends to ride in straight line more often than men.
PS: agree with Jay