Thanks for the input guys. I will try out these ideas and see what happens. I also have a seat post saddle combination I haven't used before, the seat post is laid back so I have the saddle at an extreme of adjustment which make the whole assembley flexi. Also had a problem with the saddle slipping backwards until I popped in some brass shim, less than elegant.
So a lot of experimenting.
Interestingly doesn't feel like the frame flexing. I know what this feels like as when I load up my cyclocross bike for touring with the weight down low in the paniers and not in hard contact with the rack you end up with the same shimmy. However even though the end result is the same it's build up feels entirely different and definitely something that builds from the frame itself vibrating.
My other non wobbling race bike, a Klein quantum pro, is set up so that when I go down hill I natrualy have a leg on the top tube. Never thought this was significant before.
Now lets see, Wheel bearings, headset, leg, saddel, post, getting my weight back on decents, stem length, possible front fork swap, I'll know the answer in about 2 months. I let you know what I find.