I have Wald folding baskets on both my bikes, and am very happy with them. They are bolted on, so theft is not a problem -- and they are always there if I decide to go shopping or similar on the spur of the moment from work. The shopping bag style panniers just clip on and off, so you need to remember to bring them or allow anyone who passes by to slip them off the bike when it is parked.
Wald has a poorly-designed clip to keep the baskets folded up. I just use bungee cords to keep them folded up when I'm not using them. No one has ever run off with the bungee cords, BTW. The Bungeed Walds don't make noise in my experience, and the bungees are useful for securing the load when the Walds are in use. The size of the Walds is perfect for one large grocery bag each.
I also use 20L dry sacks that I place in the Walds instead of using rear panniers when I tour. The Walds have enough extra space for a water jug or miscellaneous stuff that I don't need to keep in the dry sacks.
The biggest issue is that the bike (a Trek 520) doesn't handle well if you put too much weight in back. When the weight gets up to 50 pounds or so, the bike is awkward to steer. So I got a Soma Porteur front rack that accommodates odd lots up front (I've brought a watermelon home from the supermarket on it.). If I put 20-25 pounds up front, I can load the back with all the weight I'm willing to pedal without having handling problems. And the Soma will take a pair of front panniers if you do any touring.
Obviously, weight weenies would not opt for any of this...