If you have the opportunity to borrow a CX and a MTB, you'll likely have your answer by the end of the rides. Try each and see which you like the most.
I'm very curious to do some MTBing and have a friend that has offered to loan me one of his bike, but I have yet to take him up on the offer.