I don't have any bungee cords on my panniers, so this may not be applicable. My Frost river canvas panniers use leather straps and buckles.

To give the panniers some shape and support to help keep the pack bags out of the spokes, I put a piece of corroplast (plastic cardboard) on the inside of the panniers. Maybe a little rigidity would help in your case too?
It weighs practically nothing.