For the handlebars, the type of brake's you're interested in are called "TT brakes". They were originally designed for time-trial race bikes, but are also now appropriate for bullhorns on the street. Tektro makes some decent tt brakes.
You should definitely consider getting foot retention. Velcro straps with BMX pedals will be fine for casual riding. In fact, some people prefer them for their snug and comfortable fit. If you have BMX style platform pedals, i'm pretty sure it will fit velcro pedal straps.
If you wanted to, you could experiment with risers and drop handlebars. For a casual bike, its fun to experiment with everything!