Safe is a relative term. People ride fixies without any brakes and many people have ridden bikes with only rear coaster brakes. IMO, it's less safe but you can adapt your riding to help mitigate the increased risk.