Well, don't ride it like that!
Check all the frame joints, one of the rear stays could be loose at a joint, or cracked.
For it to be rubbing on a stay, then spring back to place when at rest, something major is amiss, if it is not a slipping QR.