a) Some like lots of pockets, some don't. I like one main big pocket, one flat or expandable or mesh side pocket and one smallish back pocket (eg. Tioga or Como brand panniers) - great compromise between useful size and handy organization. I would HATE large numbers of tiny pockets all over the place.
b) Avoiding spoke interference - a major problem with many panniers. Floppy panniers, and those with a single lower attatchment (ie. most) seem to be worse.
c) Reinforced corners - that's where they wear through, esp. rear inside corner against the spokes and rear outside against the ground.
d) Secure attatchment - nothing worse than going over a bump and having the bungy thingy lodge in your freewheel or cassette.
e) Strong and abrasion resistant straps - it takes a lot of force to lift panniers on and off the rack, esp. with elastic attachments.

My cheapish Como panniers have served me reasonably well commuting, but have failed on (b) to (e) above with heavy loads, especially non-bulky heavy loads for some reason.

I'm thinking of getting a pair of Ron-D-Swan panniers next (http://www.rondswan.com/). Otherwise maybe Crosso panniers. Have a look at both of these for some more ideas.