This is NOT correct. Or at least it wouldn't be correct if the bikes were from the same family. You're right about the seatpost, wrong about the stem.

You are forgetting that the seat post is not vertical. When you go from 54 to 56 seat tube, the point where the seat tube meets the top tube is moved up and backwards. Assuming equal angles, the switch from 54 to 56 would move the junction up 19 mm and back 6 mm. There's a good chance that angles aren't equal either: bigger bikes generally have more off-vertical seat tubes. Going from 54/74° to 56/73.5° would move the junction up 18 mm and back 10 mm.

Since the top tube is measured from that junction to the top of the head tube, this backwards move "eats" some of the extra top tube length. So, instead of 20 mm you only have to deal with 10-14 mm.

Generally speaking, TT length is a bad metric. It's easy to measure, but it's too confusing and easily misunderstood, even if the top tube is horizontal. It's almost totally worthless when the top tube is slanted. There is a better metric called "reach": horizontal distance between BB and the top of the head tube. It varies a lot less than TT length. For example, on a Trek Madone 5.2, the difference in ETTs between 62 cm and 52 cm frames is 64 mm. The difference in reach is only 19 mm: the rest is eaten by the move of the junction point.