Stack numbers are frame stack and that ignores how many spacers might be under the stem of the bikes you have tried. So you can't just go by numbers if you don't know all the other numbers to add up.
I'm 5' 11" on a 56 cm Tarmac and wondering if I might have liked the 54 cm better. Wish I'd tried a 54 cm out. Since my height put me squarely in the overlap of the 56 or 58 cm I only tried them. However I'm happy on my 56. Just one of those grass is greener things.