The only way that "wobble could be a problem is if it happens while you are actually pedalling. A bit of wobble when the chain isn't moving is immaterial.