If you do take off the rear brake and the frame is nothing special, file off the braze ons so they don't cut your knee into muscle and an artery making it so you can't ride a bike for almost a month.
I didn't file the braze ons off my polo bike, and I can start LIGHTLY riding on Wednesday.
If the frame isn't some random mehhh frame, keep the gears on it.
I used to be against taking the braze ons off.
Once I am able to play polo again, they are coming off and then I'm taping over where they were,