I've used a backpack for my entire 3+ years of year-round commuting, mainly because that's what I started with. I carry a company laptop back and forth most days, and I had seen other people (not even bikers) carrying theirs in a backpack, so I figured that's what I needed, and my company graciously bought me one instead of the standard-issue briefcase-style bag. I too ride an upright hybrid, and I haven't noticed any balance problems or so forth from it. But then, I have nothing to compare it to. I'm sure that the pannier people have a good point about lowering the center of gravity and so forth, but it hasn't been a problem for me.

The biggest disadvantage I have noticed is the back sweat issue. However, here in Maine, that's only really a problem in the summer. It hasn't become a big enough issue for me yet to go to the trouble is switching.

On the plus side, I think the advantage of being able to carry it around with you is a big one.

Not an argument for or against, but one odd thing I've observed in myself is that it feels kind of like wearing a seat belt, so when I don't have it, I feel less secure. Totally psychological, of course, but there it is.