Weight was probably around 20 pounds (single pannier) to 30 pounds (two panniers). I normally don't ride fast when the bike is loaded, but on this route there was a long downhill slope where I picked up speed. It was there the swaying usually manifested itself. I felt like the rear triangle was wobbling from side to side.

I knew it wasn't the wheel because 36H 3X wheels are as lateral stiff as they get, and the panniers are very securely locked to the rack too. In hindsight I should have suspected the lightweight alu rack, but even before I had ridden this new bike with any weight on, I had some preconceived notion that it would be slightly wobbly under load because it was a light racing/audax frame, not a touring rig. So I just assumed it was the frame that was the problem.

It wasn't before I read "tsl"'s comment I realized I hadn't had the same rear swaying this season where I have used a Tubus Logo steel rack.

--
Regards