Happens all the time to me on the Hudson River bike path in NYC. Soon as I detect someone on my wheel (spidey sense tingling), I talk to them, warn them about sudden stops because of people on the path, etc and continue.
To date, only one guy has come close to hitting me from behind. Usually the guys who latch on have pretty decent skills and I've had one guy latch on on several occasions ....

Personally, I would never draft behind a stranger.