Just a thought -- I never bring my work shirts home, I take them to the dry cleaner across the street from work. I don't wear them at home, why bother storing them at home and schlepping them back and forth?

I use a cube-wall hanger that holds two weeks of dress shirts, suit, blazer, etc.

I commute with just one pannier, which gives plenty of room for slacks, laptop, socks and underwear, etc.