The symptom you are experiencing would indicate that perhaps the wear is not even. The fact that it is not always there or in the same place would further indicate uneven wear. The other possibility is that the fixed cup is not fully tightened, and is shifting on occasion. I would say that if there is actual pitting (rather than smooth wear) it should have been replaced. As a side note, a properly worn in bearing runs smoother than when new.