Oh boy, I smell trouble.
Not sure what area you're in, but around here it's really never been an issue for me. My usual trails are unique in that ped's don't have the right of way--bikers & ped's both yield to each other. This is unusual, & at most trails the ped's will have the right of way over you, & if equestrians are allowed then they have the right of way over everyone.
When they notice me & start to move, I say "excuse me" &/or "thank you" & I might ask how their day/hike is going & give them a smile. The fact that they're outside in nature & not at home being lazy already makes me like them. If they don't move, I trail behind them at a distance comfortable for both of us & wait for a clearing where I can safely pass. I've never had someone come at me head on & not move, but I always slow down to a walking pace so that if I need to dismount & move, I can.
It should also be noted that my trails are tight singletrack. If you're on fire roads or something & they still move to the side that you call out, then I can understand why that would be bothersome.