I've ridden 32s for the gravel events I've done. It's mostly fine, but I find myself wanting more traction on high-speed descents. Otherwise, they're fine. This is on Vermont roads, which are on the smoother side, but still no joke if you venture onto the occasional Class IV road.
Originally Posted by
HTupolev
If you add a pound to your rims, and you pedal at 200W, it'll take you in the rough neighborhood of 30 seconds longer to ascend Alpe d'Huez, or a quarter second longer to accelerate from 0 to 25mph.
Wider tires should have a bit larger aero profile as well, but we're looking at a fractional increase in a fairly small minority of total aero drag for bike+rider; unless you're width-matching tires to fancy aero road racing rims (where extremely tiny changes in width can cost a few watts, and where the presumption exists that you got the wheel specifically to avoid that loss), my opinion is that it's not something very worth worrying about.
I have a bike with 53mm tires, wide but otherwise built like road racing tires. Cruising the flats, I haven't been able to distinguish a performance difference between it and my Emonda ALR with 23mm Bontrager R3s.
If you use tiny rims with wide tires, the bulbous inflated shape of the tire can be flexy about the tire-rim interface, resulting in a squirmy ride. 40mm tires on 20mm internal-width rims should work fine, though.
I wouldn't consider a 30 second difference up a climb small. Especially if attributable solely to one piece of equipment. The aerodynamics aren't a trivial concern if you care about performance.