Only you can say what you need, but I have definitely not found it necessary to use a substantial lock or to remove and take my stuff inside with me. I have gone coast to coast twice and done a bunch of other long tours and have never removed and taken my panniers off to go inside a store, not even once. I also never used more than a minimal lock. I have wheeled my bike up and down the aisles in a few walmarts and grocery stores and I have parked it inside a few stores out of the way in the front of the store.

My lock is a cheap little 5 ounce cable lock. That said, if there is a place where I feel the need for a big, heavy, expensive U-lock, I just don't leave my bike unattended there at all. That is a little easier if you avoid bigger towns and cities. On the rare occasion when I do find myself staying in a town like that, I typically get a room, but it doesn't happen that often.