Any of the Rixen and Kaul fittings are excellent. They release and attach in a second or less and are very durable and secure. R&K make bags but also sell the mounting system to bag makers such as Carradice. The Ortleib mounts are very similar.
I use a single pannier with no balance problems. It leaves the rack top free for a U-lock or large loads (eg 6-pack of kitchen towels).
If you want a rack-top bag, they make an adapter for any rear rack to take their quick-release system. This is more versatile than the Topeak system and you can remove it if you want.