I've not had a problem when I say "on your left" when passing with someone pulling to their left, although I'm sure it's happened. I have seen more than one instance where a club ride was starting to move along and someone passes on the left, or right, and doesn't announce their presence and the cyclist further forward starts to change lanes. I had a close one last week and I've seen more than one this year alone. So yeah, I announce I'm passing.
If the double pace line is well formed and for some reason someone drops out of the line, no I don't announce I'm moving forward, that's undertood.