You don't have to blow the lights to overheat something important. I would keep the same voltage.

If you can solder the connectors, look into buying or making a 6V RC battery. 3.2Ah is pretty big, but a quick search turns up some NiMH batteries with similar specs: http://www.all-battery.com/6v3800mah...connector.aspx