I did the backpack thing for a good solid 8 months before I finally moved on. I say avoid the backpack if you can.

I'd add something new to the mix and offer another solution that hasn't been suggested yet, and it is actually my normal daily commute set up: Topeak Explorer Rack and DX rack trunk w/ fold-down panniers. It has a shoulder strap you can clip on. the MTX system means it slides right off with ease when needed yet snaps into place securely. I keep my lunch, wallet, keys, phone, etc. in the trunk part, change of clothes, lock, and some other odds and ends (towel and shower supplies in summer, e.g.) in the fold-down pannier. That still leaves me room for the occasional grocery item I have to buy on the way home as well.

I do also have a pair of panniers which I sometimes throw onto my front rack for added storage space on days when I need to bring home bigger items from work, drop off dry cleaning (like today), etc.