I don't see why downsizing for a track frame (under particular circumstances) makes so little sense to people?

I'd say there is a slight advantage in downsizing just a little bit (a cm or two) if you're going to be riding the bike primarily in the city... if you'll be using it on the open road... I wouldn't bother downsizing...