Nobody, you, the driver, the peds, go hurt, so you can't have done much wrong.
I would have done the same thing, signaled, waited, that you did. Likely I'd have been far enough out in the lane that if the car wanted to pass he'd have had to drive into oncoming traffic to do it. But you didn't get hit, so I'd call that good. SOmetimes cars will try to pass you at totally inappropriate times, and that's just the way it is.