That happened to me last year at about 35 mph. Scarred me, turns out I didn't have my stem tightened down far enough on my fork, there was a bit of wobble. I tightened it better and the wobble is gone......
I've heard others here speak of a wobble having something to do with harmonic vibration. One poster awhile back said that he would touch his knee to his frame and the wobble would go away...........