Folks 'round here are pretty good, but I've extended my longest finger or the "(both arms up) WTF?" look on more than one occasion. No one has ever stopped, or come back to chat. I believe this is because I only use it when it is very obvious the driver has put me in danger, and I think they realize that.
I would extend the same gesture if I was driving a car, and I refuse to act differently because I am on a bike. Perhaps I will regret that at some point, but for now I have my dignity and self respect.