Invest in good shorts if you haven't already, and keep riding. Your speed suggest comfort bike to me, which are anything but comfortable over long distances, but they can be helped with a saddle replacement. If this is the case, find one that isn't squishy. A lot of people like the Brooks leather saddles and I think that they are well applied on a comfort or hybrid bike. Of going to a road bike is an option for you, consider it. You will find your average speed increased by 50% in short order. If not an option, consider narrower tires that can handle higher pressures, this will help some. I believe that an increase in speed will help a lot. Look at it this way, if you are riding 2 hours at 10 MPH you can go 20 miles. If you can increase your speed to 14 MPH, you are going 28 miles. Makes those longer distances easier to achieve since it sounds like your saddle comfort is the current limiting factor.
"It hurts so good..."