Dude, cars slow me down on my commute FAR more than I slow cars. Do I get annoyed? Yes. Do I get violently upset with the stupid @#$%s who aren't aware of other road users?
Not really. So I say you're in the right: it doesn't sound like you did anything other than not give-up your right to use the road in a fair manner. Good job staying cool!