YES ,.. leave it be, all of it, grind that dacayed vestige. You won't be degrading it. Besides, it needs to come-off either way. Touch it and other spots-up.