I'm curious, why are you so scared of going downhill? Do you not trust your equipment? if you don't ever go over 20mph (I ride faster then that on flat), I think you are missing one of the best points of cycling.

With all that braking, you are also killing all of your momentum. remeber physics? you are turning all of your potential energy into heat (friction from your brakes) instead of forward motion (velocity) I'm not saying that you have to hammer at 40mph down hills, but you can just coast, and carry a bit of momentum into the next roller.