I agree it's kind of annoying. and it's all mental. I used to get super annoyed with roller bladers and peds on the brooklyn bridge and would yell or whistle loudly. Then I let that go - and would get annoyed at other cyclists for harassing the peds!! ridiculous, but true. My goal is to give up being annoyed at everything I have no control over. cars are the thing i struggle most with. may never get over that one.
by the way - i am not always successful. I almost got in a fight with a guy on this beautiful colnago the other day when I passed him on the bridge and he re-passed me with a little f-you swerve that caused us to have significant contact. long story short - i was annoyed and followed him to a red light where we had words. should I have let it go? yes, but I was bigger than him with a crappy bike and feeling frisky. he offered some hollow threats that I knew would come to nothing since he would risk laying down his amazing ride. he actually told me he was going to "punch me out" which I hadn't heard since grade school. anyway, that sort of stress ain't helpin anyone.]