For me, the whole point of going on group rides with strong riders is that it forces me out of my comfort zone. The only way to get faster is to ride faster. And the best way to, at least for me, to ride faster is to ride with faster riders. I get dropped by the 1s and 2s all the time, but that day that the group shatters and you look around and realize you've made the first selection and you have no business being there but I'm gonna hold this wheel to hell and back.... that's a great day. And I still get dropped now and then, but that feeling keeps bringing me back.
As always, YMMV.