You can cicumvent the BB height being lowered 4 mm somewhat by getting shorter cranks. 165 or 160mm should be good to go. Since you're going fixed gear and only need a front break, grab a tektro dual pivot in long reach. They're not expensive at all.

I'm still a little iffy on this filing the brake arm thing. <- Unprovable opinion