Everything that has been said here is good advice, especially the Low Rider. I find that the low rider style panniers really stabilize the bike.
One more thing you can do is to get a frame or top tube pack. These will usually by small, holding a pound or two, but what I like to do is use them as a permanent place to keep a tube and some tools, and maybe an extra pair of gloves, like a full finger glove, if you are out in colder weather. Any weight you put inside the triangle doesn't seem to matter much as far as the stability of the bike.
Howard