Hmm. Could the battery pack on these new lights be 1s2p rather than 2s1p? That would make them more like the 18650 flashlights we're fond of. That would explain why the plugs are different -- the light head may not (or may) be compatible with the higher (double) voltage packs.

Measure the voltage on the pack when fully charged ... is it 4.2 volts or 8.4 volts?

If the pack puts out a maximum of 4.2 volts, then doing it with USB would be easier. But still, you'd need a circuit of some sort to limit the voltage to 4.2 volts. You could rely on the protective circuit in the battery to do it, but that's not really wise -- if it ever fails, it would ruin the battery and could make it catch fire or explode. Though if the charger really does put out 4.5 volts, then they must already be relying on the circuit in the battery? Perhaps to save a dollar or so in the charger?

Either way, I'd say that making your own charger is somewhat risky.