For headlight, look at Fenix.

I like the BC30 V2.0, which uses two 18650 batteries. The batteries have to be removed (quick and easy) for charging. Batteries are available that have their own USB ports for charging, or you can buy a charger that can handle 18650 batteries.

Other options from Fenix include the BC26R (single 21700 battery) and the BC21R V3.0 (single 18650 battery). Those lights have USB ports for charging, but the batteries are still user-replaceable.

For all of these, you can remove the light while leaving the mount on the handlebars.

I am not aware of any current-production taillights that are designed around user-replaceable Li-Ion batteries. I use a Magicshine Seemee 200 V2.0, which has a built-in Lithium polymer battery; not designed to be user-replaceable.