Thanks for all the feedback on this... I was a little on edge due to the fact that I had my little kids with me and that It was my first time on a trail, first time in the sport for that matter.
It was a bit discouraging to have to have words with the guy in front of my kids, and discouraging that it is part of their first MTB trail experience as well. Just wanted to be sure I wasn't missing anything here.
I will just chock it up as an encounter with a random A-hole and go about my business. We did have a lot of fun.