I'm gonna go with the "with permission" camp. I generally never draft unless riding with someone I know well. He and I take turns pulling, and we both know each other's riding quirks.

I had a guy on a folding bike drafting me on my commute. He made a disappointed comment when I turned off.