I can't take my bike in the building, so my road bike stays at home, and my CX is my commuter. I've taken the CX bike on gentler mountain trails that seemed to be made out of gravel. The road bike has been through short stretches of it, and on grass, but ... it's most fun going at high speeds on good pavement, and making steep, winding descents.
Also, I have to go quite a bit out of my way to hit a dirt trail in a park in Seattle, and most of them don't allow bikes.