Many moons ago, I pulled a BOB trailer cross-country behind an aluminum road-race bike. I used some high-spoke-count wheels but did not yet realize the importance of a quality rims.
The handling of my setup was pretty floppy from the start but I didn't know any better so I kept going. The cause was discovered in Ohio where my rear wheel gave up the ghost. The stronger replacement rear wheel cleaned up the handling and that was the last I thought of it on the trip.
I never used that bike with a trailer again but did break the chain-stay within a couple thousand miles of finishing my tour. I'd guess the events to be connected but that particular frame had a reputation for breaking chain-stays so, who knows?
When I think about riding that floppy wheel up and down the Appalachians...

Thank you, Lord!
So, to summarize:
1. I'd want a frame that is not weakened by flexing.
2. The rear wheel is at least equally important.
3. Pray regardless.