Sounds to me like you did a helluva job avoiding him. In situations like that most people on two wheels get in trouble because they don't anticipate the possibilities in the first place. I've been down hills on a bicycle at speeds up to 55 mph in the past but that is a thing of the past given the lack of sight lines on almost all roads where I ride and the poor performance of bicycle brakes. As I motorcyclist I learned long ago that people "don't see" you and it's up to me to have a way out of any situation.
Your encounter with him was also handled well IMO. I would however go to the police and tell your story and make them aware that in his present physical condition he continues to be a threat to others. Make the report formal because once it's in writing the police know that if he hurts someone else and it comes out that they knew of his impairment someone is in deep shi*t. I'm betting there's a means to require he be vision tested by the State DMV. What bothers me is that this guy has admitted he has a problem with vision and continues to operate a machine that often kills people. He cares more about his convenience than others' lives. If he's stopped now someone else may have their life saved. Your conversation with him may have been amicable but he is not a responsible person. And he's a danger to others.