The downside of the approach listed above is that now your road bike is super uncomfortable for 4 hour rides and climbing hills suck with short cranks, so your rides are shorter than normal and have to be sorta flat.

This is all assuming that your road work is simply training for the track. If your road work is important for group rides and road/crit races, then you have to come up with a compromise, build 2 bikes, or simply focus on fitness gains and not so much neuromuscular gains.