Got both. mtb on slicks with hydros makes a fine tourer/brevet bike. Its fast enough, useful when the gravel shows up, and can handle single track. But, on a fine day on a good road the toad bike can eadily average 7kmh faster wuth thecsame effort. Of course after 200km everything hurts.
If I had to pick one it would be the mtb, but a cx bike would be ideal to me with only one bike. Sonetimes I want to play in the dirt, or tour 400km, or ride a brevet. mtb everytime. But if I want to ride fast for two hours....