No mysteries here, and no pseudo science.

If you are starting from a place of not riding hard much, then doing so will make you faster. You are still exercising regularly and making yourself fitter and stronger. There are lots of ways to build cycling speed and endurance. 1. Ride pretty hard, pretty often 2. Ride medium speed, A LOT, 3. Ride extremely hard for short intervals. All of these will make you faster, but at different rates and in different ways. If you can ride 100 miles easily, then obviously it's going to help you ride 10 miles faster that if you've only ever ridden 10 miles.

BUT, THE SCIENCE REMAINS: If you ride with a fast group regularly, you will get faster. But if you also do interval work, and recovery rides, and fuel intelligently, you will be EVEN FASTER.

----

As an anecdote...I began riding regularly in spring of 2015. That whole summer and fall, I was riding with a fast group, and increased pretty fast like you probably are now...but always ended up with the "B" section of that group...the fastest guys were always leaving the rest of us behind. you might compare those fast rides to an "8 out of 10" effort...I was never sprinting, but I was working pretty hard to keep up.

Over this past winter, I didn't do rides like that, only some 30 minute slow outside rides in the cold, and a session of 20 minute indoor interval sessions of all-out 30-second sprints (1 or 2 times/week).

On our first few spring Fast group rides, I was able to keep up the front guys (triathlete, iron-man types) and ended up being in the front "A" group....even though I rode less over winter than normal. There's no question that those short sprint intervals once or twice a week conditioned my legs and body in a way that riding "mostly hard" had not done...I noticed increases in ability within a week or 2 of starting them.