An emery board for filing fingernails works well.
Or a bit of very fine sand paper made rigid by a popsicle stick or such.
Lightly sand down any high spots, back and forth in the direction of rotation.Stop when the high spots/roughness is gone.
Don't worry about any low spots.
Last edited by Homebrew01; 04-14-16 at 06:33 PM.