Heel clearance might be an issue with rear panniers and 20" wheels and most panniers. I'd think front panniers and some gear on top of the rear rack might be a good way to go. If you go rear panniers they may need to be small and/or be shaped to maximise heel clearance. Some discipline in packing light would allow lower weight and volume and make the packing easier and the riding more pleasant.