i'll only have a problem if someone brakeless smashes into me when they can't control the bike. as long as you can control your bike and you know what risks you're taking it's no problem. i know a lot of fixed riders who can control their bikes a lot better than some with 2 brakes.