I've bike commuted both ways, backpack and no backpack. Here's some things I noticed:

1) Backpack is easier to get on/off quickly.
2) Backpack can be used with any bike.
3) Backpack puts more weight on butt.
4) Backpack needs more frequent washing.
5) "No backpack" puts weight on bike, not back/butt.
6) "No backpack" more comfortable.

I had no balance problems wearing a backpack, even with 15 pounds inside. When I strapped the backpack to a rear rack, I had temporary balance issues until I got used to it. Then I went back to wearing a backpack, and had temporary balance issues with that, too. I got used to that, also. In the end, it wasn't whether or not I wore the backpack, but getting used to the new weight distribution.