Lots of good racers have this anxiety.
Unfortunately crashing and racing are intertwined. Sometimes there's nothing you can do to avoid it, but there are things to mitigate or reduce the likelihood of a fall.
1 - get to the front as early as you can.
2 - stay there.
Sprints - I'm always jawing during the jostling telling whomever will listen to "keep it fast", "go faster or we'll get swarmed from the pack", "go, go, go", etc.