I can't stand people holding my wheel when I'm out riding alone. I'll usually ignore them or say something. Depends on my mood.

If someone passes me I don't try to grab their wheel. The point of a solo workout is to ride alone, not suck someone's wheel. If they slow down again it irritates me.

If you manage to pass a group you probably don't want to be riding with them anyway. If they pass you and you're not feeling social, sit up for a couple minutes so you don't get sucked in.