I went brakeless with my first fixie, and I'm still alive. If you want to go brakeless eventually, why not do it now. Just take it easy for a while until you feel comfortable with it. You will become a bicycle ninja much faster than if you had a break.