I've used this setup for overnight tours so far, and I'm pleased with it. Back rollers up front attached to a rear rack that's 'creatively' bolted onto the fork. One pannier is for the food/kitchen, the other is for clothing and a sleeping bag, so the weight on the front is not too heavy. The repair and maintenance stuff is in the seat bag. The bulky stuff on the back platform. Front and rear racks are the same, ie, interchangeable. They are $15 alu racks and don't sway. Handling is slow at first, then you get used to it.

I've done some major hill riding with this setup, with a 12 hour day in the saddle one time with no pain from handling the front wheel with panniers positioned in this manner. No shimmy; it's a real solid 80's mtb frame. Keep in mind that I'm young and fit.

edit: Notice that I'm not using drop bars. The riser bar gives me more leverage and I find it comfortable.