Aerodynamics suggest that if the cargo is "drafting" off your body, you'll be faster (argument for the rack); different riding positions will cause different 'drag spots', enabling areas off the rider's back that are aero-efficient (argument for the backpack).

A lot of the replies here have to do with perceived comfort, and this has an effect, as well. A more comfortable rider, at the same intensity, will produce a smidgen more power, simply because he/she isn't diverted by DIScomfort.

AANNNN-NNDD, of course, there's another alternative, where it is that I land...it's an interesting mental exercise, but I really don't care. Simply, I prefer the weight on my back, where I can use it as my own, as body weight for body English. Faster is irrelevant to me, as "faster=less ride time" for a given journey. I'd just as soon be on the bike a little longer.