You may be able to get away with disconnecting the brake arm from the chain stay (frame member) or removing the brake arm from the hub all together. I've never tried it and can't imagine the physics of it right now but try it and let us know.