I too tend to carry as much stuff as I have room for, so I have to restrict myself in pannier size I have a set of smallish panniers and recently moved down to a much smaller handlebar bag.

In handlebar bag: wallet, snack, first aid kit, cell phone, keys, Swiss Army knife, mini pump
In R pannier: work clothes & shoes
In L pannier: spare tube, patch kit, tire levers, multi-tool and 13mm wrench (old Raleigh, takes a weird size); jacket; U-lock if I am going to be stopping anywhere; lunch.