I ride with a cammelback pack to work - about 1hr each way and that's fine with a moderate load, but for a day in day out tour - no way. I definitely don't want any weight on my back and frankly that's what all the silly looking bags attached to my bike are for.
I do carry a Serratus Genie climbing pack in my panniers so I can go hiking and buy groceries when I need to. It folds amazingly small and works well as a pack when you need it. For those near a MEC Serratus is no longer and MEC is clearing these packs out at $39. They are a GREAT item for travellers and when they are gone they are gone. I know the Calgary store has a few left.