You shouldn't have a problem off road unless you are unusually heavy and/or start doing jumps and hammering over rocks, etc, instead of around them. The only components really at risk are your bike's rims. I'm 210lbs and use a cyclocross bike with 35mm tyres for tough singletrack, going fast. But the wheels are nice ones - lots of spokes, good rims and hand-built.

In the (very unlikely) worst reasonable case, wheels that go a little out of true can be tweaked back. And ok wheels aren't much on ebay.

Do remember to check your brakes and take a puncture repair kit!