Regardless of the legality, riding on sidewalk with a pedestrian present can invite these kinds of confrontations.

I've been on both sides of the issue, and try to be respectful as a pedestrian and a cyclist, I've found that its generally not a good idea to antagonize someone who is already mad at life, no matter how in the right you are, often times it doesn't end well.