Comfort for you is uppermost but the attendant volume and weight, not panniers, appear to be the central problem for you when transporting the bike and equipment. From my own equipment evolution and experience over the last forty years comfort can be maintained easily while volume and weight can be minimized significantly. This has remained true on short and long tours of many months duration like the Divide Ride in the US and other long trips in South America, Canada and Mexico.

Since you like to hike and i assume backpack, you are probably familiar with the ultra-lite approach pioneered by Ray Jardine over the last decade or two. I have gradually adopted some of those techniques and others to greatly reduce volume and equipment weight while maintaining a high level of comfort.

I have been touring with two front panniers only for almost 20 years supplemented only by a dry bag stuffer on the top of the rear rack along the front rear axis of the bike. My comfort on tour is quite like mine at home. I prefer it that way. With two panniers and rear stuffer only, there are no problems with using public transportation or hiking.

My panniers from Robert Beckman have an adapter to mount the panniers to his backpack base for hiking and backpacking. For more extensive backpacking during a tour I exchange the dry bacg for a frameless backpack that stands on the rear rack with the pack compression straps holding it to the saddle and the rack.

I have bored other forum readers by repeatedly stating that this arrangement extends the life of my bike and wheels by allowing me to unweight the saddle and have the lighter rear end articulate over potholes and obstacles rather than slamming into them causing spoke and rim breakage. Sermon ended!