Figure out what temps you're comfortable at. Figure out what temps you'll be sleeping at. Plan accordingly.

In the south, in the summer, my bag liner is usually sufficient. But I still have my sleeping bag with me, usually under me, unzipped, and will sometimes zip it up in the night. My trips are short, a couple of days, maybe a week. A restless, cold night wouldn't be the end of the world because I will be in a warm bed in a couple of days. But even then I have a sleeping bag, just in case. You better believe I'd have it going cross country unless I knew the temps would be staying well within my comfort threshold, and given the variety of terrain/climates, I don't see how you could know that.

For me, whatever conditions I have to bike through, it's comfort to know that at the end of the day I will find a way to be warm and dry. When I try and think about how to shave weight from my camping supplies, the first question I ask about any item is, "Is this likely to be essential at some point in letting me sleep warm and dry?" If the answer is yes, it goes back in the bag, and I look elsewhere for weight savings. Some people are a lot more tolerant of discomfort than I am, so you need to know that about yourself, but I would look a lot of other places for weight savings before I ditched the sleeping bag.