When the weather turns nice, the hike and bike trails turn into a mine field. On the path I ride daily, there is an assorted mixture of wobbly comfort bike riders making slow, unpredictable moves, road racers riding super fast and not warning anyone when pacing (if a roadie is too cool for a bell couldn't they at least yell "Hey I'm coming up on your left!"), inline skaters taking the whole width of the path, and people walking dogs without leashes.
Luckily, I haven't wiped out yet, but I generally hold my ground and slow down or stop if necessary to get around someone or let someone get around me. I wouldn't have let someone run me off the pavement unless there was absolutely no alternative.