I don't think a chain tensioner would help with this. I'm going with a little more torque tightening the axle nuts. Sometimes I think they are tight enough, but later find that they are loose. Perhaps there was a snag on a part of the axle thread and the drop out or something. So after tightening try riding lightly a few yards and then retighten the axle nuts. If you have a stubby wrench, maybe try a longer one for more torque. This has happened to me a few times too, and just a little more torque seems to have stopped the wheel from sliding.
Also I've found that one of the inner hub nuts
(#28 - lock nut for left hand cone on page 29 of Sheldon's manual) tends to loosen every time I take off the wheel so I need to retighten that every once in a while. If you hold the top of the wheel and push/pull left to right and there is some wobbling you might want to check this one.