I really dislike riding with a backpack on. I have used a rack with rack bag and now use panniers. If a rack bag is big enough for what you carry, get one that has a track system to make it easily removable and just take the bag with you. If you carry more than that, panniers are great but not as easy to deal with off the bike.