Track bikes do have a much more aggressive geometry than "single speed" bikes. Look for short chainstays and steep head/seat angles. Makes it easier to move around on the track.
As far as components go for a new trackie (assuming you fit the bike right) I figure that if you have the right gear ratio, you're half there. Put on light rims and tires and that's about it.
Anything else is just a bonus.
I'm a track newbie. Raced 6 days total, 2 of them back in the early 90s, 1 this year, 3 last year (or was it two? I forget). I have the same bike as I did then, with a much lower end front wheel (I lost my Trispoke track axle over the last 15 years).
I'm running a straight gauge steel frame (an actual track frame with steep angles and a really short chainstay, so it's nimble and the rear wheel stays planted), same rear wheel, same super heavy 2.0 straight spokes, same model saddle.
For new parts I have a triple crankarm (my old pedal was frozen in the old crankarm) on a non-compatible BB ("just tighten it more"). I changed the bar/stem to fit me better, ditto the post (to no set-back), and I'm running much heavier pedals (SPD-Rs instead of Aerolites).
I'll be upgrading the front wheel as soon as I get some axles in and can convert a road wheel to a track one.
Right now the bike is a touch under 18 lbs, and a different front wheel may take it down to 17 or less.
Other than that, the only planned upgrade I have is that I want to get a slightly smaller chainring (48 or 49 versus the 50 I have now). I figure I'll race the frame until I have a different job, and that might take me years.
Since races are short, I think weight is really important. Light weight rims/tires for fast acceleration. The track where I race is kind of slow so aero is less important. Nonetheless I want to try aero wheels to see if it makes a difference. At least I can lead out from further out or something.
Re: rentals - at the NE Velodrome, it's $5 to rent a bike. They get donated bikes - and if you get a blue 55cm Ross (the painter, not the brand) bike there, that's the frame I donated