There are very expensive panniers but also very inexpensive ones available. My first set was a pair of U.S. Army surplus shoulder bags that I adapted to clip onto a rear bike rack. At the time there were no panniers available in the USA and eventually I bought a pair from a store in the UK.

Most of the panniers being sold are large enough to hold a "standard" grocery bag which makes it convenient to go from store to bike to home. Good selection on Amazon along with customer reviews.