Why does he say your heart wouldn't get any better?
I've gone off of BP meds (Beta blockers) completely. My doc lowered the dose gradually as weight came off and my fitness improved. With each lowering I briefly had borderline hypertension and my heart rate was a bit kooky - went higher than I would expect for a given level of effort.
Now that I've adjusted I think it helps. HR climbs quicker during hard efforts and some issues I was having with cramping seem to be less.