I'm a big fan of dual purpose on travel. Panniers are no different, I like ones that convert to a backpack - Arkel's bug / GT-18bp's are good examples. I also think there's a company that lets you connect two panniers together and backpack that. Would make it easy carry-on luggage for an airplane.

For me, the big empty space pannier is good for groceries and bulky things like sweaters, but I think I like two divided main pockets and 2-3 small outside packets to segregate gear / wet clothes / shoes. I'm a fan of the Arkel designs obviously - too bad they don't make things in orange