I'll add that cornering is a significant skill that many (most?) riders lack. This applies, at lower rates, even at the ProTour level. Cornering is the key to descending; understanding cornering is the key to making moves while either giving yourself an out or taking the risk of not having an out.

Group riding skills are important too, and mainly this comes from being able to focus on more than one's own riding.

Descending is just going fast and cornering speeds become slightly higher. Most riders don't corner properly. This isn't a problem at lower speeds, but it becomes terribly important when going fast.

Cornering is free. There's no fitness involved. All normal bikes corner reasonably well (excepting some tire/surface combinations, like knobby tires on pavement). And you can practice cornering every time you drive, even every time you're a passenger (focus on what the driver should have done).

I guarantee that once you can corner proficiently you'll descend with much more confidence.