You're doing the right thing. It's not the easiset thing, but it works. Riding with, and getting hammered by, better and stronger riders will make you faster.
It's no fun getting dropped, but when you step in with the fast guys they normally don't wait for you. They know what they are doing and while it might not seem like it, they are doing you a favor. They know you wouldn't be there if you weren't serious.
I'm not ready to hang with our local fast guys yet. Most of my saddle time is being spent on endurance, but after March I will be stepping up the training approach.