If I was going to spend half my time on gravel I wouldn't want to run 32mm tires. Fatter, like 50mm will give you a lot more traction and with a lower pressure you actually get more speed out of tires on rough surfaces.