I use the large Ortliebb Rackpack, and have never clipped them to my Ortlieb Back Roller panniers. I have used two 1/4" shock cord bungees.
I have used them for 15 years and a lot of miles without any problems.