If I'm only using 2 panniers then...
On my 20" folding touring bike I mount the panniers on the front, because the steering is twitchier and the mass helps make the steering more stable. The downside is you have to maintain that they have the same weight, else the bike wants to turn to the heavier side.
On my full size touring bike I mount them in the rear so I don't have to worry about one becoming heavier than the other.