Well, I think you pretty much summed it up. :-)
Wider tires definitely have benefits regarding a smoother ride.
I would slightly disagree about you "crushed stone path" description - I have a high end carbon fiber bike that I ride regularly on a crushed stone MUP where I am, and through the wonders of carbon fiber my wrists never hurt at the end of the ride despite 23c tires. But it is an expensive bike.
And it I would certainly agree about gravel roads - I've tried riding my 23c bike on an actual gravel road, lol - it's really uncomfortable. No way I would do that without a bigger tire.
But the OP complained about rolling resistance and finding the fastest tire - from those criteria, I wouldn't personally go above 28c. Most non-race rims will accept down to 25c, though it's highly debated whether it would make any difference in speed between the 2 sizes.