Consider using a slick tyre on the rear for low rolling resistance on road combined with a tyre with tread on the front where you need grip for braking and steering.
Slick tyres are usable on gravel roads to a point. Even if both front and back are slick. I have used Marathon Supremes on sections of gravel like the Flint Hills in Kansas. Here in Scotland I found the slick rear and knobbly front set up ideal for mixed surface touring.
I had a side by side comparison on a road section with a long very gradual hill. I coasted it. My two friends on conventional off road tyres pedalled the whole thing.
Of course modern gravel tyres roll better than full MTB tyres but still not as well as slicks. May depend on the gravel/road balance.