Sometimes the problem is that a guy starts deriving most of his enjoyment from a sport by being really good at it. Then he reaches a point where he either has peaked, or doesn't want to continue to put in the time necessary to be that good. However, instead of just ratcheting back and enjoying the sport at a lower level of performance, he doesn't like facing that he isn't as good as he was, so he drops out altogether. I have seen this in several sports, not just cycling.