Hmmm... do you unplug your car battery while driving/charging? Note, that a 12V car battery can discharge down to 11V or so, but charge at 14 or 15V. Lithium batteries may not have such large voltage swings.

With Lead Acid batteries it is common to use the battery as a power sink, and smooths out any current ripples. I don't see why that wouldn't work with other battery types.

One issue you may run into is that most Lithium battery packs come with a built in power controller. At 75% charge, it should work reasonably well being connected to the battery. At 100% charge, it may tend to cut out rather than letting you discharge.

Since the charge controllers are small and cheap, I'd probably add 2 charge controllers to the battery pack. One for charging, and an independent one for discharging. That way both functions are independently regulated. For your lights, also add a battery bypass switch.