Are you training? Are your shorter rides high speed and/or high effort? Are you climbing big hills?
If your daily riding is a fairly short (hour or less, say) and moderate to low effort, there's no harm in doing that without days off. That's like saying we need a day off after taking a couple mile walk.
During biking season, I ride hard for distance both Saturdays and Sundays, and in the winter I train hard at the gym those days. I go to work on Monday, and don't so much as think about touching any kind of weight or machine, including bicycles. I also usually take Fridays off from any kind of workout in anticipation. I don't think there would be anything wrong with taking a short leisurely ride to the store on one of those days off but I just know that I'm pretty compulsive, and would end up making a fast ride of it.
For light to moderate exercise, I believe the recovery period is what's known as a "good night's sleep".
Now, if you need a day off to break up the routine so it doesn't get stale on you, that's all good too. In my mind, this is all about us doing a routine we know we can maintain, and you're the expert on what works best for you.