It would take a lot of sanding effort to remove enough steel to reduce the diameter of that part enough to affect the press fit. Razor blade first then light sanding (or careful wipe with lacquer thinner) to finish up.
__________________
My C&V Bikes:
1972 Bottecchia Professional, 1972 Legnano Olympiade Record,
1982 Colnago Super, 1987 Bottecchia Team C-Record,
1988 Pinarello Montello, 1990 Masi Nuova Strada Super Record,
1995 Bianchi Campione d'Italia, 1995 DeBernardi Thron