Could actually hamper fitness if you get a much lighter, more aero road bike and ride it the same speed/distance as you ride your MTB.

If you ride all road and would like to go faster/farther, then definitely get a road bike!