It is sometimes called the death wobble or shimmy and is a characteristic of two wheeled vehicles like bikes and motorcycles. It results from a harmonic resonance vibration characteristic of the bicycle and rider as a system. It is a lot like bridges which start to sway in the wind or when an army is marching across them and eventually collapse and destroy themselves. You can't reliably fix it. Nothing is loose or broken. It is just that the harmonic resonance of the frame, your distributed weight, and all the other doodads on the bike occurs in a speed range that is within your normal experience. It is common on downhills where you may reach the speed where the resonance occurs. Also it is facilitated by not pedaling on downhills and possibly by the weight distribution you assume by a peculiar tuck you only adopt on downhills. There are a few things you can do about it. One is to pedal on downhills. The pedaling motion upsets the resonance and breaks the self-perpetuating wobble. Another thing is to clamp your knees/thighs on the top tube to stabilize the bike and break the wobble. You could also try to redistribute the weight on the bike by how you position yourself.
But bottom line, a bike that does this with you riding will never be totally reliable. Your best bet is to sell the frame and get a different one (brand, model, size, material) to put the same parts on. It is unlikely you would be unlucky enough to have the same problem on two bikes. I had the problem on a Ti bike and sold the frame. Found another one of a different brand. No problem at all. I don't feel bad about selling the frame to someone without warning them, because the problem isn't in the frame per se. There is nothing wrong with it. The whole system just combines to make the problem. So it is unlikely another rider with their own components on the frame would observe the problem.
Here is a definitive article about the problem from an acknowledged frame design expert:
http://davesbikeblog.squarespace.com...e-visited.html. Good luck.