Panniers are better for heavy loads like water and whatnot, because they keep the center of gravity lower and closer in to the bike frame. Your wobble may be coming from a high center of gravity, and there's not a lot you can do about it. Panniers are also better than an open basket at protecting packed clothes from the elements.