Eventually you're going to want to develop some more steady power, which means threshold workouts. Not sure what kind of equipment you're using, but if you don't have any instrumentation on your bike, just know this: "Threshold" refers to that point just before your legs start burning, and spending time in this zone is where it's at as far as endurance riding is concerned.

Since you're able to do 40-50 miles without problem, I think you can start putting some of these workouts into your rides. Try this: On your next ride, spend 15 solid minutes at the threshold point. Spend some time (2 min) working up to the point where your legs start to burn, and then back off ever so slightly and hold it right there. After the 15 minutes, spend 15 minutes riding easy. If you feel good after the break, do another 15 minutes at threshold.

The goal is steady progression, and to not overload yourself. Try 15-15-15 for one week, 17-17-17 for the next week, 19-19-19. Then a rest week, 14-14-14. After that, maybe try 17-17-17-17. Etc. Before you know it, you'll be killin' it on your group ride.

That'd be a good start.