I don't mind people on my wheel, and if they ask I almost always feel it's ok. However, it's important to ASK. Some people (like myself) don't care, but many do. I usually respond "sure, as long as you don't overlap". That's really all I care about.

What's HUGELY impolite IMO is to sit on my wheel for ten miles, then drop me. If someone pulled you for a substantial distance and you're able to return the favor, then please do so.