Doesn't sound good to me. Try holding the front brake and rock the bike front to back. If you can feel a little "tick" then your headset is too loose.
If that's the case, loosen the two bolts that hold the stem onto the steerer tube. Tighten the top cap bolt about 1/4 turn and retighten the stem bolts. That should fix it.