After reading the replies, it seems to me to depend mainly on application, temperature, fitness level, distance traveled, shopping habits, etc.

In short, you're not getting anything out of this. Try both. It's a long life and you have plenty of time to experiment. Try a backpack for a year, then try panniers. See which is best for you. I'll stick with my backpack and sweaty back. I don't carry all that much. An occasonal book or stack of papers. My commute is 16 miles. Even the empty rack would be an annoyance much less a rack with empty panniers on it.