I ride 58 and I tried 52 and I think ST size doesnt matter much because I can adjust the seat post anyway.
Sure, on smaller bike handlebar is 3cm lower.
Im not comfortable on bigger bike as TT is too long (feel like my arms too short, I like to grab bullhorns).
On smaller bike I feel fine but I pay the price (toe overlap).