I often ”wild” sleep under the stars without a tent or sleeping pad, so I keep my groundsheet, sleeping bag, and bag liner in one pannier, so that if for whatever reason I have to move out quickly in the night, I can be packed and gone in seconds. Everything else stays packed and on the bike.
edit: well, it probably actually takes a minute