so, here's my two cents:
I did a long trek where portions of it were single track mountain biking, most of it was rough dirt roads, and only a small portion was on pavement.
I did it on an old steel touring bike. (early 80s Nishiki Cresta). My friend did the same on a late 70s Fuji road bike. Every other person we saw while on tour was riding a mountain bike.
One thing we found was that the lack of shocks on our bikes led to a lot more damage than I think we would have otherwise gotten. The steel frames held up fine, but I wound up breaking both racks (and i sheared off the bolts on the rear rack) and I tacoed my front wheel twice. Those are things that I think would have been less likely to happen had I been on a mountain bike.
And even though we had the widest tires we could possible fit onto our frames, they weren't nearly wide enough for most of the roads there. We would have traveled much faster and much more safely had we been on mountain bikes.