An actual pack - cells glued and wired together - is what I have and what I was thinking of. I already have a charger, so if I had to replace one of the 4/3AF NiMH packs, I might consider spending more for a single, larger pack that could run both lights for longer. About $60 all told last time I checked; for me that would be in lieu of ~$30 to replace one of the existing packs were it to die.
You are right, costs could get unreasonable, depending on what you are starting with. I dunno what Dinotte charges to upgrade the AA version to Lithium, but it is also something I would investigate as an option. And of course, light and battery technology seems to have something better and brighter available every day.
And yes, I have come full circle with my Dinottes - bought the AA version because it was off the shelf simple, and then converted them to NiMH packs . AAs remain an option should a pack die.