The hitch for the trailer doesn't go to the derailleur side, does it?
Assuming it doesn't, the bent derailleur hanger is a result of something else. I'd suggested that the grinding noise also is a result of something else.
I can't really imagine you being able to do any damage to the rear triangle of the bike even with the load you have. The only real issue might arise if you jack-knife while braking or laying the bike over on the ground and binding up the hitch.
You may be getting some bottom bracket deflection that may lead to ghost gear changes, but I would think the out-of-line hanger would be the culprit there.