I assume this bike has a threadless headset, and you said you replaced the stem. Did you properly adjust the bearing preload in the headset before tightening the new stem mounting bolts onto the steerer tube. It sounds like you have a loose headset. If this is the case, you will need to loosen your stem from the steerer, tighten the top cap properly to take slack out of the headset, and re-tighten the stem bolts onto the steerer.

Edit: I just looked at the VO stem design, and it looks like this stem eliminates the top cap, and replaces it with a new, longer bolt. This new bolt is what will be used instead of the top cap to add preload to the bearings in the headset.