The top tube would be longer, however the head tube would be taller as well. So you would be more stretched out AS WELL as there would be less drop from your saddle to the bars.

The idea with a performance oriented bike like a track bike or road bike is that it's designed to maximize the amount of power you can put to the wheels by lining everything up so all the points of contact between you and the bike maximize your biomechanical efficiency. In that light, it doesn't make sense to ride a bike that is going to tire you out by forcing you to use your energy to adapt to poor positioning.

Now mind you, I'm not weighing in on whether the bike does or doesn't fit you, that's simply the rationale for WHY fit is so important