Might just hone out the fork steerer. perhaps a brake hone, or one of those
flexible hones that looks like a stalk of Brussels sprouts.. a few thousandths,
put a shine on the inside of the fork steerer.