I have the same problem and posted about it a while back
(RIGHT HERE). I thought it maybe had something to do with the carbon spokes on my Spinergy wheels but I don't really know. It just seems very unstable at speeds above 30mph. I've just learned to live with it and use very gentle pressure on the bars when I have to make steering corrections at speed. It's fine as long as I don't make any quick moves one way or the other. Beats the heck out of me what makes it "wiggle" but you are right....it's NOT a good feeling.