It sounds to me like you're using a bike not well designed for carrying panniers...can we see a photo? Have you considered a front rack or basket? Your bike might be better aimed at front weight than rear.