I really don't care if other riders pass me or not, I'm just happy to see them out there. I have spent far too many decades as the only cyclist on the road to want to take away any of the joy of riding for someone. If I am likely to pass someone and I am going to be turning off soon, I'll slow down so that I don't pass them since it does seem to bother some folks to be passed.
I really enjoy it when I can have a brief chat with someone while passing/being passed; I think that can take some of the sting and unnecessary competitiveness out of the encounter. Once in a while I will come across someone on my way in from the hinterlands who is clearly running out of steam. I'll offer an energy bar and a wheel to ride behind if they want it. Back in the '70s and early '80s (when there were a lot more cyclists on the roads I was riding), this was fairly common behavior. I even met a person who had helped build the house I owned at the time by doing this.