I pass cars wherever there's enough space that I fit into without endangering myself or others (i.e. forcing them to do crazy evasion manuevres). That's what I'm on a bike for in the first place. I'm not riding a bike to places because it's tiresome and somewhat dangerous and rarely tops 25mph. I do it becuase it is flexible... sidewalk, bike lane, traffic lane, between lanes, on tram tracks... wherever.
I couldn't care less about the cagers' potential frustration. Motto is: "If you're sitting in a traffic jam in your car and are pissed that I pass you on a bike, get your fat ass on a bike and you can do it too."
P.s.: Actually, I do endanger myself from time to time. Look at it as an extreme sport if you like.