A machine screw head tightened into a flat surface provides a pretty good seal against anything that isn't pressurized. Oil would seep out over time but it has a lot less surface tension than water.

The threads would need to be pretty loose fitting and in addtion the screw not tightened to let it out. And surface tension in the threads would still retain the last 1/16 to 1/8 inch or so in any event. But I'll grant you that if it was loose enough to rattle then it would let water out and perhaps even act as a pump of sorts to encourage the last bit to dribble out as well as the screw rattles in the threads. Sort of like Mike's spoke trick. But if it was that loose it would need something to retain it in the hole instead of rattling out onto the ground. Still, that may be something to try at some point.