I used to ride very large frames for my 5'11" size and not being able to get lower on the bike was one of the motivations for me getting a 56cm over a 58cm Tarmac. I was coming from a '91 Parmount with a 60cm frame.
I've not looked at the geometry numbers close enough, but with the 58cm Tarmac, I don't think I would have been able to get as low as I am now on the 56 cm bike with over half the spacers removed. This was a 2020 bike also, Specialized may have changed sizing methods with the 2021's.
Another thing to think about is crank lengths. Seems that on Tarmacs, those are where they go from 172.5 to 175 mm cranks. So that might be a consideration if you prefer one length crank over the other.
It was for me. I had the 172.5 crank removed and put in 165 mm cranks. With either frame, I would have done that anyway. But if you find not enough difference in stack for you between what ever you finally decide on, then look to be sure to note the size of the cranks and other stuff and figure those in with your frame bigger or smaller decision.