I think the real problem is that the needs of a pannier and the needs of a messenger bag are simply incompatible. A pannier needs (at least in my mind) to sit firmly on my bike, but still be able to be quickly removed and it needs a stiff back and decent support to keep it from flopping around and hitting the wheel.
A messenger bag needs to sit comfortably and have good stability when on my back.
I've tried an awful lot of different things trying to get these two factors to play well together, and it just never happens; the mounting hardware of the pannier gets in the way, or the rigid backing starts to jab me in the back, the straps needed to make it into a shoulder bag get in the way when in pannier mode, or else if they are detachable, don't provide the support needed to keep the bag stable.
I've gotten to the point where if I need a simple and stylish bag for off the bike, I just take my simple and stylish bag and strap it to my rack. It's much quicker and easier that way