Slicks will help, but the best way for me to put it is that mountain bikes are simply not optimized for use in triathalons. They're bulky (create a higher profile to the wind) and the gearing is lower than optimal.

It would be like using a 4x4 truck at LeMans. sure, it could drive the route, but not nearly as efficiently as the cars that are designed for speed.

Since the bike leg of a triathalon is essentially a "time trial", it is illegal to draft off of another rider. It is you vs. the wind. Every bit of aerodynamic advantage will make a difference. The riding position is low and sleek. The bicycle is devoid of superfluous "stuff" poking out to grab the air.
So, sure, putting slicks and an aerobar on the mountainbike would get you around he course, and would perhaps be a good way to see if the triathalon thing is for you, but it would be inherently limited, like driving a jeep at an autocross.

With a total re-build of a mountain bike frame, you could create an aerodynamic machine with better gearing, but would more or less require replacing every component. With that, the time and money would be better spent getting a more appropriate racing (or tri-specific) bike for the purpose.