Originally Posted by interested

Could be several things, but when looking at the picture, I suspect the big grocery bag pannier is the culprit: It is big and bulky and doesn't look stiff at all. I suspect that it will swing out of frequency when loaded, causing a slight but unpleasant feeling of bike shimmy. The rack looks like a two-legged alu-rack without side-rails. Those can be quite noodly and cause cause shimmy a low speeds, just like an unsecured rack (broken bolt etc) can cause shimmy.

Wind gusts probably makes the problem worse, and a loosely fitted attachment system like broad hooks and an elastic bungee can compound the problem too.

So check your rack, and try to fit the pannier so snugly and tight to the rack as possible. Try rocking the bike back and forth and from side to side and watch/feel how the panniers behave; do they work as inertia systems that wants to continue when the bike stops? Do they rattle a lot and move back and forth on the rack rails? Does the grocery pannier wobble like jelly when loaded? etc.