This has never happened to me using a pannier. I have had several types of panniers and I have several racks of no particular distinction... certainly nothing as fancy as a Tubus.

A long time ago I was experimenting with cloth grocery bags to see if they would be a good replacement for panniers. That experiment cost me several spokes.

Probably a good idea when you use panniers for the first time to make sure a) that they won't sag into the wheel and b) they are super-securely attached. When you begin your commute, make sure there are no straps loose and any bungees within two miles of the rear wheel are safely stowed.