Bike Forums - panniers for mtbiking?!

Just in case, I thought my long distance cycling question ought to be placed in this forum as well.
Regarding distance offroading: Since I do loops and SAG, I haven't had to camp while living off my bike yet, so I was wondering .... if riding technical to semi-technical single-track trails are panniers are realistic option? I am choosing so far to avoid them for my first non-sagged multi-day trip, which means camping often, while riding loaded. The theory is that by not doing so I won't be too wide for the narrow track and obstacles. Currently, I will use a rack and backpack, with a frame bag and possibly a bag on the front. Without seeing what my bike will feel like loaded, I'm hoping that should be sufficient. I suppose they've been used on the Great Divide Trail, but I always thought that those s-t tracks were relatively mild compared to the ones I will tackle.