This sounds crazy but after just yesterday I'm pretty sure my new commuter pannier makes people nicer. I guess they figure you're carrying a lot so they expect you to be slower or something? Put some big ole bags on the back, and fill them with pillows, or this:
