I've never tried panniers, so I don't know if I prefer a backpack, but that's what I use. I shower when I get to work, so a backpack making me sweaty isn't an issue. Also, my backpack seems to distribute its weight really nicely over my back, so while I won't ever forget it's there, it feels lighter than it is.

I agree that a backpack is a poor choice for long distances, though. Having done several 80+km rides with a backpack, I'm keen to try something else. My lower back starts to hurt a bit at those distances, and maybe I have crummy core strength, but am thinking that losing the load on my back may help.