I was exactly the same when I came back to road bikes after a few years on mountain bikes only.

It has taken me patience, 500 miles, loss of 6kg, 2 stems (100mm and 80mm), one seatpost with zero offset and a good fitting to get to where I am now. I am now set up exactly the way the bike came in the first place but it actually feels comfortable!

The biggest difference for me was personal fitness and a stronger core.