I have a blend of 50% paved road, 25%rail trail (generally nicely packed "champagne" fine gravel or dirt), and 25% dirt roads.
After experimenting a bit I settled on 38mm Rene Herse Barlow pass, which on my 25mm rims and 42psi measures 40mm actual. I am 220lbs combined bike and rider weight (i believe one's personal optimal tire size for a given bike and condition depends on weight).
Very tiny speed tradeoff on pavement (mostly wind and weight not rolling resistance) with this rig, and MUCH faster than 32-35mm on the rail trails, and feels reaonably capable when the trail turns a little bit to sh-t.