It's the geometry, I got tired of it, and enjoyed not having it on my gravel bike, as well as the increased stability from a longer wheelbase and longer front center. So I sold my Tarmac, which like most bikes with "race" geometry and had toe overlap, and bought a Giant Defy. Longer wheelbase, longer front center, no toe overlap (size M/L with 172.5mm cranks and size 45 Fizik shoes) and still just as fast and stiff as the Tarmac, imo.
For some it's not a big deal, but if it's a big deal for you, then you just need to find geometry that works for you.