Consider getting an SQR adapter for the saddlebag. wallbike.com sells these. They made by Carradice for their bags, but should work for any bag designed to attach to Brooks saddle loops. The adapter has two parts. There is a holding fixture that bolts to your seatpost and a metal frame that attaches to your bag. The frame fits into the adapter and is easily removed. SQR = Seatpost Quick Release.
The advantage for you is the mount holds the bag back a bit from the seatpost by about 1.5 inch You can also adjust the height of the bag by where you place the adapter which also helps. I use SQR adapters with a large Carradice Camper seatbag and it never touches my legs.
Having a quick release for the bag is great as well.