This really depends on what you need to take with you and your route. I prefer a backpack and can bring lunch and work clothes in one of the several backpacks I own leaving room for extra ridding clothes as needed. I use a pannier for hotter days to avoid the heat from a backpack.

The beginning and ends of my old commute involved wheeling my bike in/out of a building and a stair step. The extra width of the pannier complicated this, whereas the backpack didn't.

My trunk stays on either way, and my phone, keys and bike stuff was in there.