I was shocked when I showed up to my first race last year. I had no idea the Cat. 5 group would be going full gas from the word "let's roll". I got dropped pretty fast. Amazingly, no one cared and people still clap as you go around. Just keep showing back up and soon you will be staying in the pack and trying to figure out how to get to the front.
Good Luck.