OP; Probably being overly sensitive. Those minor scratches are not bad enough to elevate to "regular gouges". However your approach to mitigating them is solid. Just don't remove too much material.

On the more serious side, I see a big spacer being used in the headset. A fat spacer in the stack on a threaded fork is somewhat unusual. Fork probably been replaced and not trimmed correctly to fit the head-tube length. Likewise, a Cinelli stem like this is basically a racer oriented stem for riders who like to have them bars down low for minimal wide resistance. Your configuration is way up high. Highly recommend a close check to make sure the minimum insertion warning line on the stem is not above the top of the stack...your concerns about the stem being weakened by the gouges adds to the potential concern. Just make sure you are riding safe. A stem breaking even at low speed is a nasty gouge guaranteed and on the other stem of importance...

/K