I used a pannier, but hate to have to remove it from the rack. It was packed and then I bought a grocery bag pannier which I mounted on the other side of the rack. Just last week, I had one of those positive brain fart and realized I can put all my stuff in a backpack and toss it into the grocery bag pannier. So I removed the pannier and saved me 2 lbs of weight. On days I need to carry more stuff, I simply wear the backpack and place the extra stuff in the grocery bag pannier.
I like the convience of just grabbing my backpack and taking with me without the hassle of clipping and unclipping. The grocery pannier can be folded up so on windy days I wear the backpack and fold the grocery pannier to cut down on my wind drag. I'm also in the process of finding the largest grocery panneir. As it turns out, having that big hunk of bag sticking out on the left side of the bike keep driver from passing me that much closer.