I ride a fixed Crosscheck with 38c Interwolf tires, and while I rarely take it on singletrack, I think it's perfect for touring-style stuff. I ride a lot on the canal paths around here, and many of them are like rail-trails type dirt paths. Many of them are bumpier and gnarlier than that as well. And others are paved, and the Interwolfs seem to handle that just fine.
Check out Matt Chester's site or google Rudi Nadler. Those guys know what they're doing with fixed gear bikes offroad.