It is possible for strong riders to flex a frame. You probably notice the frame flexing in other ways.

IF you are flexing the frame bad enough to rub the brakes, you MIGHT be rubbing the tire on the chain stay. On a carbon fiber frame, this is a dangerous situation. The tire rub will cut a groove into the carbon fiber. This will even happen with steel frames. HOWEVER, damaged, cut, or scored carbon fiber is VERY prone to catastrphic failure. Check to be sure. Most often, this will be on the left side chain stay.

If you are flexing your frame, making adjustments like opening up the brakes is a dangerous bandaid. You need a stronger frame.