when I got my first fixed I didn't have brakes and it was kind of scary going down large hills. the bike got a lot of momentum and my leg lock wasn't strong enough. I had to put my feet down a few times and that doesn't really work

I would say use a brake so that you feel safe.