Yeah, panniers are the way to go. You can get backpacks that feel pretty good (airflow to your back so you don't sweat and all that), but I find that even with that carrying a bunch of weight on my back can make a comfortable saddle start to hurt. Just too much weight on your tush.