What kind of bike did you ride previously? If I've ridden my touring bike for a while, and then switch to my "race" bike, the "race" bike does feel twitchy, it takes a few miles before I feel comfortable on it.
Not saying that's your problem, it's just something to consider. Also, if the headset doesn't feel right, it's possible that someone put in a new fork and didn't install the headset properly, which also means that it's possible that they installed a fork that isn't appropriate for the frame geometry. Can you tell if the bike has the original fork?