Honestly, there's a limit to how much hand-wringing about this is going to help.
1. Have you spoken to your doctor about your riding, and do you have blessing / guidelines from him / her?
2. Are you smiling when you ride?
As long as those are both "yes", the fitness, the breathing and the lower HR will come the more you ride. The best indicator of this is your resting HR.