it could be that the stem is just more flexible than the one you took off. but first, go over the steps to install the stem properly to make sure you didn't make a mistake.
is the stem a 31.8mm clamp diameter? otherwise, if you took a stem from your road bike to your mtn bike, you took a 26.0mm stem and put it together with 25.4mm bars. so even if you tightened it all the way, you may not have adequate clamping force.
just a thought.