I don't know if what you said is valid (although it sounds plausible) but I don't notice any balance problems unless the bike is standing still. Even riding through the winter and being subjected to some slipping and sliding while loaded down with school books I never noticed any stability issues. If I might offer some insight from an engineering point of view; it seems to me that the pannier being mounted so low on the bike puts it below the center of gravity, thereby not affecting the balance to any significant degree once the bike is in motion. That's my attempt at plausibility, FWIW.