I agree with the previous poster that said get a bike fit first, it is an additional expense but will give you an informed opinion from a professional regarding your concerns, blind on the internet is only going to give you very general rules of thumb, that said I think a 56 is too big!

Have you tried putting slicks on your current bike and try that for awhile to see how it feels? Road bikes are primarily faster due to the more aerodynamic riding position, if you are so upright on the road bike it really negates that advantage. The other is the lower rolling resistance tires, that is why I suggested going to slicks or inverted tread mtb tires, something that will allow you to roll better on pavement. If you google bike tire rolling resistance you will see Schwalbe Thunder Burt’s which have a bit of knobs but roll better than many road tires.

Your choice of gravel bike category is a good one as many have mounting points for racks and fenders which the majority of road bikes do not. Depending on road conditions and traffic, I like the new 650b road plus tire size from WTB, 27.5 inch wheel diameter which allows 47 c tires which makes broken pavement more comfortable and offers the ability to ride trails or on rough shoulders when that is safer.