Yes, you're going downhill but you don't start out with some residual velocity. You still need to build up speed. Your legs will work most efficiently at a certain spin rate and so you still need to cover that range with multiple gears. Granted most DHers don't run a full spread of front chainrings and some of them might not even have a very wide rear cluster either. The big front chainrings they do run are usually pretty big... as much as 60T in some cases.