The trainer doesn't have to touch the frame to cause problems. The drop outs are locked solid by the trainer while the seat and chain stays are moving/flexing from your weight especially if you do any out of the saddle training.
If you are sitting there taking it easy with minimal effort, likely no issue.
__________________
nine mile skid on a ten mile ride