I think it depends on fitness level. When i was new to the sport, i took one day off after 30 miles thrashing but eventually realized it wasn't enough after suffering leg injury. 3 years in and 1 day is good enough to recover some but not 100%.
I discovered the roller stick and for $20, that is the most amazing thing, i've spent my $$ on so far
. I spend 20 minutes on my glutes after doing two Crits or saturday 60 miles group hammering and next day i'm 90% fresh.