I agree completely. I ride a bike to enjoy the ride. I regard my bike to be more a thoroughbred race horse than a burro. Panniers, esp front panniers or handlebar bags, ruin the handling of a bike. When I had my fixie built, I didn't even want eyelets for fenders or racks (I use clip-on race blades here in the wet Pac NW). This has everything to do with center of mass and unsprung weight.

On a one-track vehicle, such as a bicycle, you want extra weight as close to the center of mass as possible. Many people think you want a low center of gravity. Not on a one-track vehicle that gets leaned over in turns. You also want to minimize unsprung weight. If your bike doesn't have shocks, then YOU (your arms & legs) are the springs, so having the weight off your bike & on you (sprung) makes way more sense in terms of responsive handling.

I also use a Deuter pack made for cycling, which has an air channel where it contacts your back. Yeah, I sweat on warm days, but I'm sweating anyway, it's no big deal. On climbs, I can make use of the extra weight on my back to help drive my hips forward to drive the pedals with my body weight, so the extra weight is actually used, it's not "dead weight" as it would be if it were panniers.

The downside is that I get some lower back pain on longer rides if I stay in the saddle too long, so I have to make sure I'm out of the saddle, stretching out my back every now and then. But the upside is that I get some strength training, riding as if I weighed 15 pounds more than I do. If I were to go on an extended tour, I might consider using a British-style saddlebag, one attached directly to the saddle. Closer to center of mass, doesn't make the bike look stupid. I would never put a rack on a good bike.

Luis