You're probably better off with a carbide bit or a rotary file in a die grinder. I've used a regular 3/8" drill in a pinch and it worked fine.

Ebay or Amazon have 5/8' carbide bits for cheap.

IMHO It will be really difficult to do with a file, although maybe a 1" grinding stone would clean things up some ?

Good luck - you've been through a real battle on this one . . .