I commute with a backpack. Previously to this, I commuted with panniers. I changed because I switched to a folding bike, and didn't want anything to interfere with the fold.

When I set up my new commuting folder, I'll find some panniers that don't interfere with the fold, and hopefully a rack that isn't too big and clunky. I don't see any good reason to add unnecessary weight to my shoulders and back, even if it's only a change of clothes.

I carried a laptop in my Jandd panniers for years, in their laptop sleeve. It's suspended from the top of the pannier, so it absorbs much less road vibration than stuff sitting at the bottom. No problems at all--I'm typing on that laptop this very moment.

If your panniers are falling off your bike, something's wrong. Did you use a rack approved by the manufacturer? Did you follow directions? As for moving the goalposts: backpacks can cause rotator cuff injury if you use the wrong one, or don't know how to fit one. There's a downside to everything if you do it stupidly. Which I do. Often.