I say keep riding with these guys until you can hang with them. You will only improve by riding with people better than you, not riding with people equal to you.