Slightly off topic. I've experienced metal fatigue in the mount arms between the rear rack and the frame when using a rack with panniers off road. Make sure your rack and all mounting hardware is beefy. All that bouncing around and vibration is hard on equipment.
I'm thinking of doing an off road trip next summer - packing light using
bikepacking methods. I won't be able to carry as much, but less worries of rack hardware breaking and packs catching on rocks and brush.