If you think you might encounter some rain while you are biking, and you don't want your gear to get wet, I recommend that you buy panniers made out of waterproof material which close by rolling the top of the panniers for a waterproof seal like a boating dry bag.

Panniers made out of nylon aren't waterproof. They leak through the zippers, they leak through the seams, and after they've been used for a while the fabric itself leaks. As Bentbaggerlen explained, if you have nylon panniers, in order to keep your gear dry you have to pack everything inside waterproof bags inside your panniers.

Some panniers are made out of waterproof material, but they have an open top which is covered by a lid of some kind. Examples are Ortlieb's Bike-Packer Classic and Bike-Packer Plus. As Bentbaggerlen explained, water can get up under the lid of that kind of pannier and get your gear wet, even though the bag is constructed of completely waterproof material.

Some panniers are made out of waterproof material, and are closed by rolling the top of the pannier and fastening it down with buckles, like a boating dry bag. Examples are the Orlieb Front-Roller Classic and Back-Roller Classic. This kind of bag will keep your gear completely dry even in drenching rain and spray from your tires and the tires of passing cars. I like this kind of panniers, because you can just toss your gear into the panniers and know that it will stay dry, no matter how inclement the weather gets. You don't need to put your gear into waterproof bags inside the panniers.

It's easier to get a reliably waterproof seal with roll-top panniers which are made of a relatively soft and flexible material. It takes a bit more care to get a waterproof seal with panniers which are made of stiff material, such as the Orlieb roll-top panniers. Panniers and boating dry bags will leak if they are completely submerged in water, such as when a raft flips upside-down. But few bikers drive their bikes into bodies of water.

Most waterproof panniers have a single large compartment. If you have been using panniers with a number of small compartments, and you have been keeping your gear organized and readily available by always putting each item in a specific compartment, you might have to adapt your organizational style if you get waterproof panniers. But the principle of always putting particular items in a specific place in your panniers works great whether your panners have a lot of small compartments or a single large compartment. "A place for everything, and everything in its place."