If someone passes me and says "hi" or something I usually just watch them go. But if someone just buzzes by me without a word, I sometimes try and figure out if they were just passing me because they are a faster rider or if they passed me because it was an accomplishment to pass someone.

The way I figure it out is by matching their speed and waiting till they look back to see if they've left me far enough behind to drop back to their normal speed, when I see the "look back", I know they had speeded up just to pass me. So now the fun begins, I stay just a bit behind them (not drafting) and keep the same speed as them, once I see the big gasps for air I know they will start looking for a side road to go down to save face. Then I pass them back, say "hi" and keep going.

If I match the speed of someone who doesn't look back, then I have used their passing me as motivation to pick up my pace and I get a better workout trying to keep pace with someone faster than me. Most times I get the "look back".