If I recall correctly, there should be an inner bolt that secures the arm to the spindle, then an outer cap that acts as a lock ring to keep the inner bolt from loosening.

You should probably make sure you have that outer cap.