I noticed that bike racers recommend structured recovery. (no discussion from me there)
I never do formal races but push my limits. As is said above, I listen to my body. There is no point in getting injured. That can be done and I have done it.
OTOH, I bike nearly every day at modest speeds. There is no need to take days off biking if you are within your comfort zone. Your comfort zone is of course very subjective.