That's exactly how it works. I normally charge the 18650 pack off a USB port of a PC over night.

I also have a USB adapter for my dynamo hub that could be used to recharge the battery back for longer trips away from the grid, but so far I haven't had any rides long enough to require that.

One other nice thing about it is that I could make it lighter by removing up to three of the 4 cells and downgrade its capacity for shorter trips, if I was concerned about weight.