I was riding a fixed-gear with plastic platforms and a brake, and even that felt terrifying. My feet were constantly slipping off the pedals and one hill in particular felt like it would toss me out of the saddle.
I switched to a bike with spiky metal platforms... a little better, but not so much. I still ride with a brake, but the toe clips and straps feel necessary while riding fixed, with or without the brake.
But then again... I'm a bit of a wimp, so whatever.

edit: I'm usually pretty live-and-let-live about bike choices, but that seems like one choice that could seriously harm people around you if you lose control of your bike, so... yeah, kind of a ****headed move.