This definitely sounds like a headset issue. Have you tightened it/adjusted it yourself? It sounds to me like it is either massively overtightened or badly worn, as suggested above, although my headset knowledge is limited to modern threadless setups. What kind of bike is it, and what kind of headset?

The good news is they aren't expensive.