Have you checked your headset lately?

I had a hell of a shimmy on my touring bike (was actually better with more weight in front), and it was completely cured with a headset replacement. Before replacement there was a noticeable knock in the headset that would not go away after servicing and adjusting.

Another thing could be stem length....is your stem really short?