I wouldn't be deterred from a frame bag if I lived or commuted in an area without strong side winds. A frame bag could be pretty handy. But my area has some stiff winds at times and it's easier to remove a trunk bag or panniers if needed. A compact frame bag with quick releases might be okay in my area.

Sunday afternoon I wanted to test some new panniers -- empty -- on an east-west route with south winds at a steady 11 mph and gusts up to 30 mph. I didn't notice the steady wind causing any particular problems but a couple of gusts felt like someone had shoved the rear of the bike sideways.