Check a couple things:
1. make sure the end of the fork's steertube is recessed about 3mm below the rim of the topmost spacer (or the stem, if it's the topmost in the stack. If it's not, adust the amount of spacers until it is.
2. loosen up the stem's clamp bolts so it's free to swivel on the steertube, then snug the top cap down just pinky-finger tight. This takes the slop out of the headset bearings. Now align the stem to the front wheel and snug down its clamp bolts, lock the front brake and rock the bike fore/aft to check for looseness in the headset.