What was the store?
I know for a fact the Kroger Brand of stores and Walmart, and most all major Grocery stores can make you leave the bike outside(unless it super busy I guess). It is basically the same as a old person with a walker or their own wheel chair in the store. As long as you don't ride it they wont care.
We had a guy on a Segway ridde it through the store and self checkout. There was nothing management could really do.
If you leave the bike in a checkout lane they might have a problem with that, and could potently move it them self. but just walk with the bike.
I lock up my bike almost everywhere but normally, but bring my bike into large hardware stores. Never had a problem, I feel silly sometimes though because the clicking of the free wheel seems amplified and it seems like everyone is looking at me. but those places are big.