I'll give you my experience and logic - and I have a trailer. I was driven largely by finances as I picked up a barely used Burley Nomad on Craigslist for $125 (pretty good deal) that I think someone left behind when a relationship ended. I'm able to put this on an old "sport geometry" bike that I find comfortable - but have since switched out the crank on it (from a road double) to a used Sugino AT (50-36-26) triple and a taller stem - so I have good gearing for touring and in a relaxed riding position. I haven't used this set-up a lot and the longest tour I've gone on was for four days (and I can't compare to a "touring" bike because I never got one although I was very, very tempted). When I considered this trailer, I did consider weight. The trailer weighs about 14 pounds...which seems like a lot until you consider that racks and panniers can easily weigh about 12 pounds or more (especially more for beefier racks or water proof bags). So, the weight is about equal, especially once you factor in that with a two-wheeled trailer, you don't need a markedly beefier wheelset, tires, or bike to handle the weight. Of course, you're still pulling two more wheels but,that, by itself (judging by empty trailer) hasn't been noticeable to me. I do prefer how the bike handles, generally, with the two-wheeled trailer compared to loaded down rear panniers.
In the trailer vs. pannier debate, more people do seem to report liking panniers...if that makes a difference. I think panniers are the more common/traditional approach to touring. At the same time, there are trailer diehards. I think it is hard not to like whichever way you go as both obviously work well enough where both have their adherents and and both do require some investment (see cognitive dissonance).
The main driving factor, for me, was that I could save a lot of money by not buying a dedicated touring bike, modify my current bike (vintage 80s sport geometry Trek which I find very comfortable), while still getting a set-up that is a legitimate touring set-up for fully loaded touring (with tent, small stove, etc.). I spent $125 on the trailer, already had the AT crank (but a new Sugno XD600 would cost around $110 with a BB costing an additional $20), $40 for the stem, and, for good measure, got kool stop pads for my brakes (still caliper, but no issues with my clearances or stopping, probably around $20). So, assuming new crank, that would total up $300+ to be good to go (on the bike I had) or cost me, personally, about $200+. This is also me doing all my own mechanical work on the bike; paying for labor would make this go up a lot. I don't even want to think how much a dedicated touring bike, racks, and panniers would cost....probably $1000 to $1600 + any other modifications I'd want. Even a used tourer (which might be the more appropriate comparison here) would still probably cost more than that. So, I've managed to tour comfortably without spending a lot of money.
Now, this being said, as a project, I did buy an '83 Trek 520 a few months ago ('83 520 is really a sport geometry bike marketed as a tourer) that I am setting up as my dedicated tourer with some unnecessary personal preference tweaks unique to what I like when I'm touring...and readjusting the old bike solely back to my around town exerciser and commuter. This bike will still be set up for the trailer. So, I'm finding a way to spend money anyway but not too much...still far less than getting a heavy duty touring bike and setting it up with racks and panniers.