Just looking at the photo, it appears that your nice rear rack may not have sufficient lateral stiffness. If that is the case, you might want to consider doing something about that. It would cut down the lateral sway. One way to do it might be to add stays on each side going from the frame attachment points at the drop-outs to the middle cross member, attaching to the middle cross member at least an inch inboard of the outside fore and aft tubes. Does that make any sense?