According to the stuff I've read, having someone drafting you should make your work EASIER. Not as much easier as what he's getting, but by being there he's reducing tail turbulence. I wouldn't care if people are back there all day, except I don't maintain a steady enough speed to have them very close. Therefore if they're trying to stay too close, I guess I'd want them to go away for that reason.
Honestly it hasn't come up. In nearly 13,000 miles, I've had about a dozen cyclists either pass or be passed by me, and only a couple of times have they been going about the same speed, and none of them tried to get closer than about 6 feet.
__________________
Work: the 8 hours that separates bike rides.