I use a trunk bag that has panniers that zip out and fold down when needed. They're not quite as large as regular panniers, but I don't pay the wind penalty when not in use either. They're each large enough for a gallon of milk and a dozen eggs, and the trunk portion is comparable. For larger grocery runs I put on regular panniers and can still use the trunk portion of my main bag, and can even stuff a bit into the partially folded down panniers, but that's just getting silly.

I've found that if I'm carrying more than a few pounds, moving it from the rack down to the axle makes a huge difference in bike handling. Just as surprisingly, I've not been able to notice any significant difference due to not distributing the weight evenly left to right, although I generally do anyway.