I carry a set of allen key bits and a small handle to go with it. They live in the top box, where I can easy access them if needed and they're secure so no one can use them to pilfer bits off my bike when I park it somewhere.
That said, I do need to replace one of the torx bits, with the aid of a cordless drill, it picked a fight with one of the bolts holding my disc rotor on and came off second best.