You are aware it's there. Just do like I do with alloy rims or damage to any other part of my bike. Just look at it a little closer every other time you think about it. If you see bad stuff starting to happen, then you can figure out if it needs replacing sooner than later.
If you ride a the levels of a professional cyclist on mountain roads with steep descents and lots of switchbacks, then you might get some different advice. But for everyday average amateur riding it's not a problem to me.