I try to put more weight in the front than in the back. It definitely changes the handling, but not in a bag way. On my last trip, after about a week of riding around with four panniers or two front panniers, I took a ride with no panniers and my Trucker suddenly felt squirrelly as hell. Took me 5 or 10 miles to readjust to my lighter front end. I put plenty of my personal weight on the back, so moving more weight up front makes it smoother for me. And, as long as you still have a rear rack, you've got nothing to lose. Don't like them up front? Just move them.

I haven't pared my gear down to two panniers yet, but I did get to two panniers and the top of my rear rack. I decided to switch back to four panniers. Partly because I want to carry cold food on my next trip, and that was going to take another pannier for the cooler, but also because with four panniers I can keep my front and rear rack platforms free for random items I might pick up or things I might want to temporarily pull out of the panniers and have access to.