Without any backpack or bag attached to my body I feel much more comfortable and free to move. When carrying a backpack, I tend to use one hand to adjust it to relieve my shoulders from time to time. Much worse in hot summer day with sweat on the back: keep lifting the backpack from the back to cool off. In short: unwanted hassles come with a backpack that can affect speed (by a little) and safety (more concern). But it obviously save time when parking the bike.