I have one of the Planet Bike blinky lights. (I know you said no blinky & no push button, but hear me out.)
The button on it got to where it was a crap shoot to get it on/off, so I cut the wires. I have 3 other lights on my delta recumbent on the top of my seat and use the PB at the bottom on the frame. By touching the connections I have it set to blink...as that's how I always ran it mounted where I have it anyway. To turn it on I open it up and put in the battery...and to turn it off I take the battery out. Unless I touch the connections again it just comes on in the blink mode when I put the battery back in. If I had it set to static mode that's where it would come back on at. And if set to off, well, it would be off. So after all of that, the loss of battery connection has no effect concerning the switch settings.
One of these days I'm going to get a small toggle switch and install it if there is room for one.
BTW, have you tried putting in some foam rubber over the batteries to try and keep them in place? I've done that before.
Bob