When I was riding brakeless I had a 46x19 ratio.. I didn't have any close calls, but then again I only rode that bike for a season before I got a new one. And really, the main reason I went brakeless was because there was some crazy clearance on the front tire (700c in a lazy 27" fork) and I couldn't get my sidepulls to work well. I was too lazy to get a good brake, so I just left it off.
With my new bike, I do have a front brake, but then again I commute more often on it, and ride 46x17. I would only dream of taking off the front brake when I visit the velodrome.. that ratio is a little too high for me to skid consistently. But then again I'm a pretty small guy.