Two solutions:
1. Work on speed. Ride 20 MPH for as long as you can. Rest a little and do it again. You'll find that gradually you'll be able to hold that 20 MPH speed for longer distances but it won't come easy. I did say WORK on speed.
2. Cheat. If the people that you want to ride with don't have the decency to wait for you to finish, why should you wait for them to start?