Sure it matters. If the condition is improved performance, then there has to be some type a plan to improve that performance. And where there is, then rest factors in to that, and the amount of rest will vary depending on age, fitness level, experience, etc.
However, if the condition is a social component, and rides are done just for the sake of riding or the enjoyment of a group ride, then that's something altogether different and the likelihood of over reaching or training becomes even less, thus the structured implementation of rest and recovery is unnecessary.
You mentioned in your op "tearing yourself down to regrow and recover". I think most people are inclined to rest and recover when they're tired already, and additional life factors cause an overabundance of rest to begin with.
Most all of the people in here could probably improve by increasing their training load. It seems you're presupposing that this "copious group of posters" have already maxed training load and would actually benefit from additional recovery. I assert that isn't the case for the vast majority, if not nearly all.