Are you riding pavement and maybe have rolling terrain where you are constantly going up or down? That 32 pound bike is probably just wearing you out. Get something down below 20 lbs.

But still, when I started back riding in my 50's, I struggled for a while just to do 10 miles in rolling terrain. Now 20 miles is just getting warmed up and my legs loosened.