On my normal 30 mile loop of the MUP, I have to ride a few blocks in a downtown area. Where I cross a bridge that's about 3 blocks long, I get people holler at me to get on the sidewalk.
I usually holler back (after I slow down) - "it's against the law for bikes to ride on the sidewalk here, it's even posted. You get on the sidewalk" as I pedal away from them.
It's really easy to beat their speed on this bridge, as traffic is usually pretty heavy, with very wide traffic lanes.