Happened to me many times. I generally ride out away from the curb in that situation, maybe even without enough room at all to pass (with time to swerve safely). This usually gets their attention and they will slow and/or swerve. Holding an arm out can help too. They don't actually want to hit you. If your arm would come within 6 inches of the car, you can pull it in at the last second and then you have a couple of feet. You have to worry about looking like you're signaling in that case though. Try to look like you're just shaking out your elbow. If you sell it they might not even think you're trying to antagonize them (it's better if they don't think that).