I ride with a backpack often. Do my commute with a backpack filled with my clothes, lunch, and sometimes even a laptop. Have done up to 80 miles with a backpack.

I find the key is to get a pack that adjusts so that it just rests in the small of your back when you're riding - that way you don't even notice the weight.