I think the two classes of bikes hide the real differences. Which are tire size and suspension. A cross bike with 50mm tires and an rigid MTB with 50mm tires are pretty equal. Throw in wildly varying tire sizes and suspension and things change pretty quick.
At Southern Cross this year I rode my 26" hardtail and absolutely blasted by some riders on CX bikes during the descents. Once the washboarding gets bad enough and the road gets steep enough the CX bikes just can't keep up with a hardtail/full-suspension MTB.
I'm not a great descender but riding a MTB hides a lot of that and allowed me to ride downhill significantly faster than the previous year. I also found it a lot more fun to descend with a front suspension so even if my placing was slightly lower than it would have been on a CX bike I had significantly more enjoyable experience.
However, it was with much chagrin that I noted most of the narrower-tired CX riders who I passed on the descent caught and passed me quite easily on the flats.
I'm not sold on either one being faster, it really depends on the set-up and the rider. For me, the MTB was faster downhill but slower on the rollers and the flats. It was also kind of a pain to manually lock-out the fork dozens of times.
I have six gravel events left this year. I'll probably ride my MTB for 3 of them that have serious descents and ride my CX bike for the other 3 that are flatter and faster.