you can get longer bolts.

a triple , for a crank with a spacer , uses a longer bolt than a double.. or the outer 2.

then you use the double nut , to get the compression of all the parts

its maybe more comforting to you to have the bolt stick out past the nut, a bit.

the classic little spanner to grab the slots in the Nut edge should allow that .