Any ideas from anyone would be welcome.

I have also had another thought, to drill a hole through the sleeve and steerer (in the pressed section) and then put a pin through it as a second measure. Obviously the pin would need to be hardened and have a high resilience to shear force but what are peoples thoughts?

Or am I worrying about nothing?