It could be frame flex, and it could be just the wobbly headset and the lumpy front tire.
I'd deal with the tire first, then move to the headset if you have to, and if those two don't fix it start investigating where it may be flexing.
That said, loaded, my longtail gets twitchy as hell.