I know, and have experienced, exactly what you're talking about. But the lactic acid buildup theory of muscle fatigue was discredited earlier in this century.
Current theory suggests that in the thick of things, calcium channel leakage weakens muscle contractions.
Report here.
Soreness after exercise, called DOMS for Delayed Onset Muscle Soreness, is thought by many to be the result of tiny muscle tears, although this remains to be proven.
Article here.
Neither I nor the authors of the pieces I linked to dispute that recovery--light spin immediately following exertion, and again the day after--are effective treatment for the two different conditions.
I'm just calling attention to the theory of lactic acid causing the pain has been discredited. In fact,
lactic acid seems to be the fuel used by muscles when we go anaerobic, which in cycling is called "burning a match".
The Matches.
The key thing is that the more matches you burn, the longer recovery takes. After burning the whole book, so to speak, it can take up to three days for younger, healthy athletes to recover. So it's no surprise that in older athletes like us who take longer to recover, that going too far anaerobic, too often, can lead to an overall loss of performance and a general weary feeling.
This is why--at any age--the focus is staying on the aerobic side of your LT. Save anaerobic efforts for those times when there are no other choices. It's very, very hard to train for higher anaerobic output since recovery--especially as we age--takes so long.
Meanwhile, I'm off to burn some matches today in my last major hill training day before the
Highlander Cycle Tour this coming Saturday. Recovery and tapering through the week...