Unless there are big rocks or soft ground, 23mm tires are fine. You can ride lots of dirt, gravel and even some single track on a road bike with no problem. And you won't know unless you try it.