Stuff that works:

Regular panniers tend to be easy-ish to carry off the bike, make good grocery bags, and can handle a good bit of weight. REI and most LBSes will have something inexpensive, small and reasonably sturdy. 30L will handle groceries for 2 people for an average week. (mine cry at the idea of 25lbs of rice tho) Most grocery panniers that I've looked at are not well made and would not hold up under a real world grocery load.

Bike baskets are *good*. They are sturdy and no one in their right mind will steal them because they're so cheap. Wald makes folding rear baskets meant for groceries that might be exactly right for you. It is hard to overload a rear basket. Front baskets often have a goofy design and can't handle much weight on a stock mount, so they might not work well for you... I am very fond of mine, but I often have light but bulky stuff wasting space in my panniers. Stick the light and bulky in the basket, and then there is space in the panniers for the dense stuff.

If your rear rack has a platform, get a cargo net. They're cheap, and they make it easy to carry weird stuff that won't fit.