A quiet word or two is my typical response when someone is trying to push me off a wheel... usually something along the lines of "uh no" or "hey man I really don't want to get into a pushing match with you" for the more aggressive guys. But, delivered in a way that makes clear, I will absolutely get in a pushing match with you.