Bike stuff gets mounted on the bike and stays on the bike. Personal stuff goes directly into the panniers. Get yourself a convertible backpack/pannier so you won't have to switch bags all the time. Practice makes perfect -- you'll get faster the more you do it, especially the riding part.

You're taking the most efficient route, yes?

-Kedosto