That is probably my biggest complaint with my bike packer plus. They just don't compress well unless fully-loaded. That and the fact that the bottom "hook" of the QL-2 mounting system is both flexible and leaves a lot of space between the inside and the back of the bag, which can requite you to wrap the rack in something like cloth tape to prevent rattle.

OP: I hear you about clothing taking up extra room. I am 6' 2" with broad shoulders. I am L or XL in everything. The extra material can add up, especially if you have to carry clothing for a range of weather conditions. Plus, you undoubtedly need a longer sleeping bag and mattress that many others. And you likely have larger off-bike shoes. I wear a size 12-13 depending on the make and model.