I had something kind of similar happen on my beater mountain bike last weekend. A spandexed out guy came up behind me and said "Morning! Nice day for a ride!", and promptly passed me before I could say Hello. I stepped up the pace a bit, and kept up with him only a few yards ahead of me for quite a while. I don't think he realized I was still there, because his eyes got really big, and he started cranking and sped off.
Now, mind you, I definitely couldn't have kept up with him when he was really trying to get the speed up, but for a while there, I was right behind him!