It happened when I was riding but I can't pinpoint an exact moment. When the wheel is off, I'm careful not to rest the bike on the dropouts. Ultimately brought it in to my local shop and had it checked out. They filed down the rough areas and assured me it is safe to ride. I'll keep riding on it for now and watch it closely. The mech did say my quick release was not super tight so I wonder if that had something to do with it.