How tall are you and more important, how high is your seat? I am your opposite. I was never fast downhill. I've gotten very good at getting aero but I still am not fast. (Racing weight -145 lb, now ~160. Was 6'-1/2", getting shorter. But long legs, high seat. (I ride long cranks so I shade toward high BBs to keep my pedals off the pavement.)

I hated guys who were shorter and heavier than me which was almost everybody.

Ben