I've had GT54's for about three years. They're great. I use them constantly for utility cycling. They work as well as when I bought them, and they still look good.
They are better panniers than the XM. The internal aluminum frame, hard floor, and large side panels make loading much more convenient than would be true with the XM. Shoving everything into a soft bag and dealing with straps on the XM would get annoying really fast. All those extra pockets on the GT are a good thing to have on tour. The fuel bottle pocket and the Thermarest pocket are not just gimmicks. They increase convenience dramatically. All of the extra pockets, not just the fuel and Thermarest, help with odd-shaped items-- wine bottles, bread, little things you don't want to throw into the main chamber-- on errands, too.
That said, GT are more expensive, as you observed. XM would be better in confined areas, like forest or bush. The GT are big, too, so you very well might have problems clearing your heels with a small rack.
If you can afford them, if they fit, and if you're not going to be riding through narrow brush trails, you'll enjoy the GT more.