I test rode a Safari and found the same problem as you. Figured it was just my short arms.
It seems that four measurements are critical to a comfortable ride. Standover, saddle height, bar height, and reach. Once the first two are settled, the last two can usually be customized with a stem riser, washers, and/or an adjustable stem. For a touring cycle, the additional weight is negligible.
If you like the XL, an adjustable stem may be all it needs. Otherwise, go for a LHT.