A brake wheel cylinder hone is what you might have to use to get it smooth as it should be. I think Lisle makes one, and it's available at most decent auto parts stores. Make sure to keep the stones as far down as possible, it would probably do a little paint damage if it came out at too high a speed. And make sure the lug looks round, and the gap in the rear not tapered narrower at the top. Should be a rewarding not too expensive repair. Something that should have been done during frame building, IMHO. I have more than a few bikes where seatpost gouging is a problem, so I need to follow my own advice.
,,,,BD
In my defense I have been playing with my aluminum bikes lately.....