Yes, but it's not quite that simple.

A 6v SLA battery will actually put out about 6.6 or so volts when fully charged. Four alkaline AA cells will start at 6.0 volts and go down from there. (Four NiMH AA cells will start at about 5.6 volts and go down from there, just to cover all the bases, even though 4 x 1.2 volts is 4.8 volts -- NiMH and NiCd cells start at 1.4 volts fully charged, even though people call them 1.2 volt batteries.)

It's likely that the SLA battery will be just fine -- 6.6 volts is pretty close to 6.0 volts -- but the voltages aren't exactly the same.

Actually, there is one other possible problem. It's possible that the light puts the batteries into parallel and works off of 3 volts instead of 6 volts. Or perhaps it leaves them in series to get 6 volts, but also connects to the batteries somewhere else so it can get 3 volts. This is unlikely, but it's certainly possible. I've got one of these lights, but don't quite remember what the wiring inside looked like.