Definitely loose headset. Get a headset wrench and tighten the cups. It's not the stem and fork, just the headset. Or if you're not into spending money on a headset wrench, most LBS's will charge you a fiver if that at all.