This old track coach used to recommend that new riders first get their skills down pat through lots of bike handling drills: low speed stuff like shoulder, elbow, and hip bumping, tire touching, track stands, as well as "silly" drills like picking up waterbottles, riding backwards, no-hand trackstands. Additionally there would be on-track stuff like "follow-the-leader" on the banking, pacelines and reverse pacelines, two-up pacelines, and whatever else could be thought up.
Then work on basic tactics like jumps, pins (the "razor"), taking runs, etc. as well as "chalktalk" about the tactics and rules for the various races.
Then come the pack riding drills, most of which involve nothing more than getting used to moving through a pack at various speeds without killing anyone.
Then the training races, with an emphasis on tactics, positioning, and simply understanding the rules, including the gray areas.
And finally the racing itself.
IMO, any rider pursuing an interest in the track should at least learn all the races and get some basic exposure to them all, hopefully advancing a category or two in the process. Then he/she can pick a specialty.
In some cases, I have had riders who absolutely insisted that they wanted to do nothing but pursuit or kilo and didn't even want to ride in a pack. That's not ideal IMO, as racing really is very effective training. Great gains can be made behind the motorcycle, but I always felt that regular racing was part of a complete training program.
In some cases I had riders that were such godawful bike handlers that I didn't want anything to do with them riding in a pack and would only allow them to pursuit or kilo if they were going to work with me. Hopefully you don't fall into that group.
Pursuit and kilo, BTW, have turned into two utterly separate events in the last couple of decades. The training is much, much different and I'd tend to discourage someone from trying to do both unless that person has no particular concern for results. (Which is just fine, IMO.) Training for pursuit will put you in very good shape for typical massed-start track events and much road racing. Training for kilo will make you fast as hell but will likely not give you the endurance you need for pursuit, points, etc. and will make a mess out of you if long road races are your thing.
Short version: Yeah, if you really don't want anything to with massed start races, then it's "allowable" to do nothing but pursuit or kilo training. Then you just need to stay disciplined enough to not get involved in other people's pacelines, packs, etc. as it's very irresponsible to endanger other track users with your incompetence. Maybe you could wear a reflective vest that says "Fred" in big letters or something.