It depends on the situation, like others above have said, on whether or not I'll announce I'm passing.
I usually don't though. When I do, its done verbally.

I don't expect others to let me know when they're passing. I use a mirror, keep my line, and stay aware of whats around me.

No, I don't find it annoying when they don't announce they're passing ^.