Riding fast alone isn't any more safe. You're less likely to be seen as a soloist. In the case of a non-fatal injury you can assured you will have your entire crew with you calling 911 and getting you medical attention ASAP. Everything has its trade offs, but you shouldn't stop yourself from doing what you love because there are risks involved. Even getting out the door or showering has its own risks.
I hope he wasn't close OP, either way losing a member of the group is a hard feeling you never forget and I've experienced it more than once. My group does honor rides for our fallen comrades; I have one upcoming this weekend. Sometimes when I'm sad and it's raining I'll go on a ride anyway; the rain hides all tears.