Alcohol, food, you name it. Both of those have direct impacts on my cycling abilities even days after I've consumed them. A steady consistent diet lacking "pollutants" is a good way to ensure consistent biking ability. Oh, and days off. DEFINITELY days off.
For me, days off don't affect me a lot. Today was my first day on the bike since Thursday, and I was rockin'. Felt great, speed was great, etc etc. YMMV, but generally that's how I work. I'd think it's likely more the intake during that period than anything else.