I've only had this happen to me once. In all honesty, it made me uncomfortable. Not because I was afraid that he would hit me or anything, but more so that I pissed him off and he was angry that I passed him. It was on my relatively short 5 mile commute, down a long and straight road that's probably a mile long. I passed him on the slight uphill leading to the long straight road. I then proceeded at my normal speed after I was a good 30 yards in front of him. I was probably going 20 MPH at the time. (I tend to fly on my short commute.) Half-way through the road, a car comes up behind me and I see two shadows instead of just my own. (It was dawn, he didn't have a light, the car did.) I look back and the guy is right behind me, drafting me. So, thinking that maybe he was mad that I passed him and he wanted to go faster, I picked up the pace. I probably ended up at 25 MPH or so at the end of the road. Luckily he went left and I went right. If he would have said something, I wouldn't have cared, but because he didn't say anything, it made me uncomfortable. I don't often have people keeping up with me on my commute so seeing him was a bit of a surprise.