You ride plenty to be in shape for you tour. Rather than focus on fitness, focus on the comfort of your bike. Change the riding position such as the height of the saddle and/or bars. Get a new saddle, bars, and/or stem if needed. Wrap that bar with new tape, get new tires, etc.