Backpack, hands down. The reason I choose to use a backpack rather than panniers is that I never know where I might find myself going (on-road or off-road) and I prefer to not have anything hanging off my bike so that I can maneuver more freely if I decide to take a more scenic route home. This has converted several friends to ditching their panniers as well as they found them to really be a PITA when we ride through tight traffic situations or divert to a tight trail ride, because their bikes handle a fast pace over bumpy country roads or through heavy traffic much better. If all you are going to do is pedal around on smooth surface streets by all means use panniers, but if you may find yourself riding more aggressively or over more mixed terrian, a backpack is the way to go.

My personal favorite for the past several years has been The North Face Bandwidth backpack, and I wear it almost everywhere I ride, hike, or climb.