When I was a kid we went into some camper's tent and rummaged through their stuff. The camper saw us and chased us away.
That's the kind of stuff that would more likely happen. If I'm going to leave my camp I put it up where others can see it rather than in some secluded spot. I think it's less likely to be messed with if it's out in the open.