I have done tours with only rear panniers and the bike handled just fine. On my last tour I used a front rack and put most of my heavy stuff in a small pack on top of the rack leaving only my light bulky items for the rear. I liked this set up a lot and will be using it on future tours.

Bottom line every bike, rack, pannier & rider combo is different and you are going to have to try whatever you want to use out and see how it works for you.