Don't let your sleeping bag get wet. Problem Solved.

E.g. make sure your shelter is reasonably waterproof, and put the sleeping bag inside a plastic bag.

It won't rain every single hour of every day. You should have plenty of time to air out the down bag if it gets a little damp.