The rumor you hear about sizing is due to the bb height. I say they only ride a smaller track frame so they have better clearence to touch in the street, but if you're riding on a track, you aren't going to be standing over your top tube all day. 54cm is 54cm.

PS- size it for the top tube.