I should add that I am only considering Endurance bikes with disk brakes and clearance for a minimum of 30mm tires, and comparing these to similarly specced Gravel bikes, e.g. compact 50/34 front chainrings, carbon frame.

I'm assuming it's mostly down to wheel clearance (up to 40mm or bigger on gravel bike), frame geometry (maybe more relaxed, longer wheelbase on gravel bike) and all up weight.

Sounds like there may not be that much difference in reality and much of it will be down to tire choice and bike fit.