I can see why people get pissed and I agree that they are entitled to their pissyness and totally justified. It doesn't mean that you shouldn't do it if you like it. It is reckless, dangerous, and people will get annoyed, try to stop you, or call the cops but it doesn't matter. Base jumping from skyscrapers and bridges is also a reckless and dangerous urban activity that also happens to be illegal and is bound to piss people off but that doesn't mean base jumpers should stop jumping off of buildings. Just remember, the guy who does get annoyed or hostile is less likely to behave properly for the next cyclist he comes across as long as you are cool with that, do it. That's why I tend to grab vehicles driven by people other than their owners. The driver of a city dump truck seems less likely to get angry than a dude in a sports car.
As for touching cars other than skitching; If I don't know you, keep your hands off my ****ing bike. I'm sure car-owners feel the same way.