Whether you're pedaling or pedding, there's nothing wrong with reasonable assertiveness. Put one foot in the street, and raise your arm with your hand held open signalling a "halt". If that doesn't do it, let it go. The next guy will probably stop.
The guy who blew by probably was in a hurry or something. Give him a break. Don't have a coronary over it.