In my experience, the ones who yell are not the ones who actually do anything about it.

I had a guy throw an empty Coke bottle at me once. He never yelled. Just tossed it at me as he went by in the left lane.
The other time was when someone actually spat in my face. I was in the shoulder, minding my own business and not stopping up traffic at all. A guy pulled up next to me and the passenger stuck his head out the window and spat a mouth full of water directly into my face (glasses, mouth, helmet, etc...). they sped off at 50mph before I could get a license plate.

The ones who yell are just being bullies, but in my limited experience, they are all talk. The ones you have to look out for are the ones who don't want you to see it coming.