Unless you do all your riding on dedicated bike paths and flat, traffic free roads, I would keep the brake. Pedestrians and drivers do too many stupid and unpredictable things that usually result in you getting hurt in the encounter. Cycling in traffic is dangerous enough with brakes. Why add more risk to the equation?
Some will post that "risks are the jam" "brakes are for fakes" whatever. I already know that I am not the jam, so **** taking risks. As far as being fake, I have a massive disinterest in what others think of how I roll, so I have a front brake.
Do what you think is right.