A file is too rough. 320 might work, but 400 or finer is slower for more control. Wet it with water and wrap on a small sanding block. I can buy one sheet of wet-or-dry sandpaper at the local hardware store, like Ace Hardware, or probably at an auto parts store. Those sheets are handy to have around the house, anyway.

I don't try to get rid of the pits at all, I just smooth off all the raised edges around the pits. The block can be a piece of wood or even a small plastic box with a flat surface.. It's surprisingly easy to sand a an unwanted dip into aluminum, as I found out many years ago when trying to smooth out a rim seam that wasn't quite flat.

I've gone over new rims that had a rough milled surface with some 400 grit, folded a few times to a narrow strip, pressed with a finger, and spin the wheel on the bike. I'm careful to stay only on the brake track.