There is some statistical evidence that too much exercise can cause some issues. E.g. older members of the 100 Marathon Club -- runners who had done over 100 marathons -- had signs of fibrosis.
However, it wasn't clear this in turn led to significantly higher rates of cardiac disease. Other factors like genetics, diet, general health and stress will play a part. It's basically impossible to determine an individual's threshold for excess exercise.
Unless your doctor says otherwise, I don't see any particular reason to worry about it. Just get your regular checkups, eat right, maybe check your blood pressure regularly, and be aware if you experience an arrhythmia.