Any cables pulling the bike to one side? I guess not if you are running a single speed. Deffinately have a look at the headset.