You've listed the pros and cons well.

Backpacks are very popular among bike commuters here in NYC and even more so with bike couriers, which are plentiful here. For the couriers, they save time. Another advantage is that when you use a backpack (or messenger bag), your cargo is sprung weight. If your bike is lightweight, this improves the ride. Attaching heavy cargo to a light bike makes the ride much worse. I have a lightweight fixed gear bike (about 22 lbs), and I decided to keep it without any cargo carrying ability.