It takes two to race. It's only a race if you let it be. I just crank at the same speed, and pass or get passed, I don't care. A couple of times I've overtaken people on race bikes (on my cheap hybrid) and passed them, then they passed me. A mile later they'd run out of steam and I passed them again. One of the guys passed me again, and I was catching up to him again when he turned off to another road. I sort of wonder if he really was going that way...
These same idiots probably do the same thing in their cars, can't stand anyone being in front of them.