You know what pisses them off even more? Acting like they're not even there. I do this all the time. Partly because I sometimes ride with headphones on (low volume though - I do want to hear horns, people shouting).
Take the lane and know that you own it. Don't even acknowledge them. You wouldn't if you were in a car. Then act like you're f@#king crazy and not worth messing with. Act crazier than them. I disarmed a situation once by saying complete gibberish and nonsensical stuff. The other party did a bit of a double take then decided it wasn't worth it.
That's when you rip a maniacal laugh!