I'm 5'11" with a 34.5" inseam. The sizing guide for my Tarmac gave me the choice of a 56 or 58 cm frame. Since I knew I wanted to have a lower and more aero body position, I went with the smaller size.

Frame stack, which affects the height of the handlebars is lower as the bike size gets smaller. Reach is also affected. But not to as large of an extent. And that can more easily be dealt with by shorter or longer stems.

Without going to weirdly angled stems, riser bars and steerer tube extenders that to me will make the pretty bike a fugly bike, your choice of handle bar height is more restricted and only the 40 to 35mm or so of spacers that they give you with a new bike. So for certain, get the size that puts the handle bar at the height you will want.

And realize that the new bike is usually set to the max handlebar height the designers intended. And should only go lower from there.