Same thing just happened to me (yesterday actually) on my new ss old peugeot. Pressure from downstroke pushed my wheel against the frame. I couldn't tighten the QR enough and it kept happening.
I found a little piece of hardware that had been on the bike that I had put into the midden pile of cables and derailleurs--it was a small piece of steel that had been inserted into the drive side horizontal dropout. It looked like it served to block the axle from slipping further back--to keep the axle pushed up into the middle of the dropout.
I took that and and slid it in front of the axle (now all the way back in the dropouts), which held it firmly in place. I don't know what it's called, but it came off an old Peugeot UO8. I'd think other older bikes would have something like it. Sorry if this is a bizarre or inaccessible solution.
Others suggested to me replacing my Quick Release with a more solid axle so I could crank more on the tightening, or, as I see above, knurled washers. Good luck.