I've had a few bikes with this problem, some much worse than yours, and rode them for a long time with no trouble. I'm no expert on what frames can and can't withstand but, if it were me, I'd ride it without worry.