It's not so much YOUR weight because that's a live weight that can shift up onto the pedals. It's the dead weight of the load in the panniers mounted on the rack that has no effective sort of spring to it so the pannier load hits the wheel like a dead blow hammer with ever bump.