TTT,
You and I are similar, so I'll take a shot at an answer. I'm 45, 6'2", 225 lbs., and have been riding for four or five months after five years of inactivity. I ride 60-90 miles a week. Overall, we've pretty close.
No, I don't think you're going too hard. First, I'm assuming you have a healthy heart. If you do, your body won't let you hurt it. Second, if you can sustain 20-miler at a decent pace (which your HR-monitor confirms), you are maintaining elevated HR for an hour or more, right? That's another data-point that says you're not going too hard.
As you know, when you burgeon into the anaerobic zone, you're beginning the slide to conkville. When your muscles burn on a sprint or climb, yes, there's some anaerobic respiration going on in there, but once you sit down and regain your normal pace, your leg muscles to aerobic again, and the blood flow flushes out much of the lactic acid the anaerobic bit just produced.
The slide between aerobic and anaerobic is not an absolute line. It's a gray area, and some muscle and subsets of muscles are anaerobic at the same time others are not. So use common sense as your guide. If you can sustain a strong pace for 60 minutes, I don't think you're going too hard. If your legs are sore or tired the next day, then go light that day to recover.
As we get older, the adage "listen to your body" is a good one. If you have any doubts about heart health, see your physician for a check. As you keep this up, you'll be dropping some pounds, right? You'll get fast and faster. Have fun!