most of the time, I'm sure the majority of us tour in summer months, so summer bags are warm enough.
We've also taken small fleece blankets with us on some trips, nice on cool mornings or nights with a summer bag, with fairly minimal weight and a bit of extra volume, but with pannier traveling, there's usually room for them.
I wouldnt take my down quilt on a summer trip, it would be total overkill and not pleasant to sleep under or even loosely over parts of me, so having even an inexpensive, small summer bag has its place in our outdoor stuff depending on the weather conditions.