I am surprised to see how many posters experience that kind of thing regularly.
I agree with others who have said it is not cool and even can be dangerous to draft a stranger without even notifying them. That said, it's possible the guy just didn't know he was doing anything wrong.
I suppose the solution would be situational. If there's time, maybe take the opportunity to educate the drafter (Of course, the OP didn't know the guy was there until he was about to turn.). Otherwise, I'd probably just slow down gradually until he got off my wheel.