You have to work them both, particularly because we're already well into the season. Speed over a century is all about endurance. The only way to build endurance is mileage. But a fast century is also about speed. So you also have to do speed work, and it takes time to build up speed work just like it does to build endurance. Though you develop speed faster through speed work than you do endurance through endurance work, the problem is that for a fast century you need to develop the ability to ride hard for a long time. Think 80% of MHR for your average HR.

To ride a century, you should have done several weeks where your weekly mileage was at least 100. For a fast century, your weekly mileage should be around 200 for a few weeks. Some of those 200 should be at well above your century pace, including some hard hill work, and some rides should be at below, even well below your century pace. Every week you should have one ride of 60-80 miles, including some good hills, and some of those 200 mile weeks should include century length rides.

So you have to start easing your weekly mileage up and at the same time including some rides at high intensity. Don't try to ride long rides at high intensity at first. Separate the two endeavors. Shorter high intensity rides, and long, lower intensity rides. Gradually increase the intensity of the long rides as you are able, without putting yourself in a hole and not being able to keep up the mileage for the rest of the week.