I know that caffeine seriously effects the way my heart operates. It does terrible things to the way it beats and I wouldn't be surprised if many of our fellow cyclists who push themselves well beyond normal limits, suffer similarly but are not aware that caffeine may be the culprit.
As no studies have been done that I know of, to assess the long term effects of caffeine consumption in athletes, I'm taking what I believe to be the prudent path and limiting my intake substantially. I mostly drink de-caf now and I've never felt better in my life.