Well I feel differently than most about an aluminum frame, I think it is better than steel for a touring frame. It's lighter, stiffer and does not rust. The so called harsh ride is because of the stiffness, but to me that means an efficient bike, very good energy transfer from pedaling. A comfortable ride is easy to get by component selection, a steel fork, Brooks saddle with springs and generous size tires. I get no harsh feel from my touring bike at all, even on 5,000 + mile tours. (The whole welding a steel bike on tour thing is a touring legend and myth)
I weigh 225 and hard as I try, I can't seem to pack light. That is part of why I use a Bob trailer, most of the weight goes on the trailer wheel. I have never broken a spoke. I think with panniers I would be breaking spokes often. If I did use panniers, I would go with a 40 spoke rear rim.