There are various ways to do it. A coaster brake has a clutch with a sleeve around it; the sleeve provides enough drag that when you backpedal, the parts of the clutch get closer and push two shoes out against the hub shell, or compresses a set of discs, half of which are keyed to the hub.
Freewheels on derailler bikes can us pawls, little bits of hard metal hinged at one end and pressed out by springs. The outboard ends go into notches in the freewheel body. Pedal forward, the pawls engage. Coast, the pawls become stationary and click as the notches go past. There are also ramp-and-roller freewheels: turn the inner part, the rollers go out on ramps and engage the driven element.