Don't know yet as I don't have my frame bag yet, but
AFAIK pretty much everyone keeps food and water (i.e. heavy stuff) in their frame packs, including on rough terrain. In a rackless setup, that's the best place for heavy things. Second best would be at the bottom of the saddlebag, I guess.