You increase tension with a screwdriver, turning it clockwise.
The nut is PERHAPS supposed to be used as a locknut, so you tighten it while holding it in place with a screwdriver.

The only thing I don't like is the fact that screw could screw up the leather, it doesn't have a wide, flat head like the Brooks patent. I would put a regular, flat head screw instead of the original one.