Yes, lock them up and make them not easily ridable. That and perhaps an old chainsaw left out with some fake blood on the chain as well as dribbled all around a workbench with a spare mannequin leg dressed up to look like it's been freshly cut off. Place this in easy view of a window for when they peek in.

I had a neighbor who kept an old mannequin in the back of his truck with the arms and legs duct-taped together and a bag of it's head. Next to this he kept a pick and a shovel.