Well made panniers have quick-release, locking hooks that hold the pannier firmly onto the rack, and a stiff board within the pannier to hold it's shape and prevent it drifting into the spokes.
Cheap panniers have poor fitting systems and lightweight materials that are prone to spoke contact. The best solution for these panniers is a rack with a dogleg strut.