Joe,
After graduating from college, where I played rugby all four years, I joined up with a local men's club. Their website said "all players of all skill level welcome." So I went, played for two weeks, and then one day during practice we did this drill, and I nearly passed out. I messed the drill up because my group was one man short, so the whole practice came to a halt. I left after that practice, and never went back.
And now, four years later, I don't think I could ever play rugby again, and I regret it. That probably wouldn't be the case if I had gone to practice the day after my "disgraceful" end.
One of the guys on the team emailed me a few days later and asked where I was. I never replied to him, and that was the end of rugby for me for now, unfortunately (God Willing, I'll get back to it some day).
Get back on the horse! Don't let it get you down!