Stick 20 or 15 lbs anywhere on a bike and there's no way it won't affect the ride and handling. Personally I like front racks but that's just me.

The other comment is that not all backpacks are the same. Something with a proper harness is a lot more comfortable and cooler than something that just happens to have two shoulder straps.