Other than the KK Rock and Roll Trainer, does a rider using their bike on their trainer harm the frame?

I don't stand on my trainer, but what i'm saying is, does stress build up on the frame because the bike is (almost) completely restricted from moving side to side? The parts of the frame that might be most effected are the dropouts, seat and chain stays.

I don't know about others, but when i pedal, especially with larger gears for intervals (on my trainer), the more intense i go, the more i can feel my bike flexing, but this might be due to the material of the frame...and i know that the bike is installed on the trainer correctly.
My frame is aluminum, but i also know that is stiff on the road.

Soo...is it harmful, and to what extent?