Building a base?
 

I've been doing a lot of riding and at moderate to high intensity. I've also been reading about building a good base however this will mean riding at less than 80% of max HR which is about 145bpm for myself.
But the speeds this is done at will be a lot slower and the riding easier.
It just doesn't make sense that riding at 25kmh for 2 hours is going to help at the next club ride. What am I missing here?

It's about feeding your muscles. On your club ride, you'll burn a mix of fat, glycogen, and digested carbs. Moderate riding tries to move more of the energy production toward fat, so you'll conserve glycogen. Then you'll have more endurance on multiple hard climbs. Plus moderate riding does a lot of other things that can also be done by higher effort riding, but at a lower cost in training load. When I look at my time in zones for a club ride, I'll see a lot of zone 3, then zone 4 time somewhat less than zone 3, then zone 2 time less than that, and just a few minutes in zones 1 and 5. The theory is that you want a pyramid of zone times, broad at the base and narrow at the top. So during the week, I try to create a more sensible pyramid of time in zones by concentrating on zone 2 with a little zone 3. I fill in zone 1 time with recovery rides and hiking, though my zone 1 time is always less than my zone 2. If I'm recovering well, I'll also do some zone 5 work during the week to fill in for what I didn't get on the club ride, if I need to.

This is from a perspective of doing hard club rides or races every weekend. A long time ago I asked Pete Penseyres what I needed to do to be more competitive on the 3rd mountain pass of a ride. He said, "More base." He was right.

I think I am understanding this. This will enable me to burn fuel more efficiently whether from fat or glycogen?
I will try tonights ride, which is a recovery ride anyway, at zones 1-2. But its windy, and that will mean slower speeds. I find it hard to ride by my HRM instead of the speedo. Maybe I will just ignore the speedo.
I can't get it out of my head that I should be riding harder all the time to get faster. But I will heed your advice.

On a recovery ride, you should be embarrassingly slow. Hence they are almost always done alone, preferably where no one knows you.

The ride went pretty good and I found that an easy 25kmh is possible with my HR around 115-120.
But I had a guy pass me and stop three times. I think he was doing it just to see if he could catch me again and again. Other than that I didn't mind the slow ride at all and I think I will do two of these a week.
And my HRM said I was burning way more calories than fat. Usually it reads between 15-25% but last night it was much higher. I think 75-80% if I remember it right.

Things are going ok with this.
I did a 40km time trial this week and although it was partialy through a thunderstorm and some wind I still managed a 1:15 on my road bike with no aero gear. I felt great during the time trial.
I am getting used to other riders passing me when I am in recovery mode. One guy even rode with me for a bit when he was finished an interval. He completely understood.
I want to work intervals into my other rides. How can one do intervals in windy conditions?
The speeds are lower and will the intervals be shorter?

The interval times shouldn't be any different. The distance will change but you should keep the time an intensity the same.

After two weeks of building my base the weekly club rides have become a lot more enjoyable. I can hang with the faster groups longer and I caught a few riders during the climbs. I seem to have more "reserve" now than before.
Maybe this is just a mental thing but the results speak for themselves. I am going to stick with these base rides as part of my routine.