I did most of my track racing on relatively relaxed frames. Especially for someone specializing in longer races, "road" geometry works fine.
So the short answer is that the bike the OP envisions will be just fine for his purposes - but so will an aggressive "sprint" geometry and a relaxed "road" geometry.