I toured with a backpack 7 years ago because I don't like racks or panniers. I use a trailer now but when I use my backpack, my pack with gears weigh 7-1/2lbs. The only issue was sweat on my back and that was enough for me to dump my backpack. If you don't mind the sweat issue, I don't see why it won't work for you.

I've seen some guys that have 5lbs of gears when they go on a backpacking trip but I don't think I can do that. 7lbs is the bear minimum for me.