Tubus makes a Great Clamp on mount to do what you need Done , buy a set of those . They are nicely machined .

In general the Boss reinforces the hole , and the trick is to really clean and flux the metal and then Heat it with the torch Just Enough to melt the brazing filler.

dirty hinders molten brass flow, and minimal heating is best.

part 2, HAND FILE , buy a nice new sharp file. grinding takes off metal too fast for precision.
and a file is like a saw, it only cuts in one direction, It's not like sandpaper.