I think Avner just was venting and he said he was embarassed, so maybe just wanting to air it out.
Personally, I don't think you did much wrong. I have been fortunate in that I've only had one guy yell at me. Funny thing was that we were at a 4-way stop and I did a stop and go, sort of a rolling stop, expecting him to come to a full and complete stop. Well, he didn't. He didn't stop at all, just slowed down. He yelled out his window and I yelled back. Stupid on my part, as we were both technically in the wrong.
The thing to keep in mind is that you don't know when you are going to run into a real fruitcake, possibly packing a weapon, etc. You can't be too careful.
I think you handled it perfectly by telling him to drive on. You have to stand up for yourself. Nobody respects a wimp. You weren't confrontational or profane, you didn't give a rude gesture or anything. The dude was obviously just wanting to push your buttons.
Where it really turned bad was the swerve. Personally, I think if a driver swerves to attempt to contact a bicyclist it should be attempted manslaughter. What else would it be? I'd be inclined to get his tag number and call the police and you had witnesses to back you up also.
Bottom line, you're o.k. You could have gotten hurt, we're all thankful that you did not!