Baskets are super convenient IMO. My wife's favourite bike has a giant Wald newspaper delivery basket and this can carry my two largest panniers as well as another ten pounds of loose stuff. Baskets that attach to the sides of a rear rack like panniers are also very practical, esp. the folding type.

The disadvantage is that they are generally not easily removed, so your bike is immediately a few kg heavier. Panniers are generally quick mount/release and so you only have the added weight of the rack (>1kg) that stays on the bike.

If you have only one bike and use it for running errands and for recreational riding, then panniers are a better option. If you are planning on having a bike dedicated to carrying groceries and whatnot, and keeping a separate unladen 'recreational' bike then baskets are great.