You are never too old to start track racing. I started racing on the track at 38 - only two seasons ago...
Racing you are grouped in grades so race against people of similar ability. Also you can race Masters events too so race against those of similar age.