We shared the same goal, keeping up with the crowd. For me it was group rides, there's many paths here's mine.
Rode alone until I could do 20 miles regardless of speed. I then joined the nearest "C" ride (C being slower than B or A). For me this meant a 60 mile round trip drive to and from the ride. I went from barely hanging on to easily completing. This required me to ride 2 or 3 days during the week to get stronger. Next step was a big one, the "B" rides are much faster and further. Good news was the B rides are in my town. Currently I'm a front of the pack B or back of the pack A rider.
Ride lots, mostly fast (fast being relative to you) and you'll get there. Mainly though have fun!
Welcome to BF