This is okay as long as the bag is clean. When they're dirty (as a pannier near the wheels is wont to be) then you have to carry around something that gets you dirty as well. I can see why people don't like to do it.

I've done little commuting by bike, but when I carry stuff I usually use a messenger bag (I currently have a Timbuk2 classic), which for me works well for light to medium loads. I hope to do some commuting to one of my jobs this summer, we'll see how well the bag works then. I've never been fond of riding with loaded panniers (the few times I've done so), although like so many things it's probably something you just need to get used to. If I did more utility cycling I'd probably look more into these things. As it stands the bag and maybe a rack are all I really need.