I tend to agree. As a Machinist, I feel that what you are observing with regard to the groove is a combination of displaced metal and paint, which worsens the defects visual condition.

The base metal (tube) is of lesser damage that it appears IMO, thus not compromised. If currently all tightened up with the inclusion of a new headset and bearings, I'd ride it as is.