IME the three enemies are weight, volume, and wind resistance. The more volume, the more weight because larger containers are heavier. Keeping every thing behind the rider lowers wind resistance, but may compromise handling. One has to experiment. Except for the panniers, light weight hiking gear will help with the weight, but not the cost. I first concentrate on volume, lower volume items also being lighter. Another approach is simply to buy lower volume and thus lighter panniers and just use gear which will fit in them. And yeah, low gearing rules. Many fun places to tour also feature climbing.
__________________
Results matter