From what I've seen, there are lots of theories, and precious few actual solutions. Some claim too little trail is the cause, others say too much trail; too much frame flex is the culprit, or is it too little?. Heavy wheels seem to cause it, but light ones can make it worse. FWIW, I've had stiff-as-a-board frames that wobbled, and willowy ones that didn't; low-trail bikes that were rock-steady, and high-trail ones that shimmied if you looked at 'em funny. Even weight distribution isn't a consistent issue. My 'Traut Limited tourer wobbled with a rear load, but was rock steady with the same load up front. My Gunnar Crosshairs was a regular hula dancer with a bar bag.

Word is that headset friction may be part of it: more friction seems to damp things out pretty well. So tighten up your hs maybe 1/8 turn (or less), or install a needle-bearing unit. Cartridge-bearing headsets have the least friction, so may aggravate the problem.

There's been a lot of attention to the issue in BQ lately, and one theory to come out of that is that the distribution of rigidity within the frame matters. A slightly heavier downtube and lighter top tube seems to work better, which my experience bears out. Most of my favorite bikes have been made from the old Reynolds 531 db set with 1.125", .9/.6/.9mm downtube and 1" .8/.5/.8 top tube, and in retrospect those bikes also had the fewest wobble issues. Of course, that doesn't help much for an existing bike...

SP
Bend, OR