I'd put one of these under your spacer, It will keep the fork in the frame.
http://www.bikeman.com/HD9974.html QBP item, can be ordered thru your LBS.
With a Integrated headset, there are a bunch of small parts that fall out when the fork steerer is not clamped , with the stem .

so then after fitting this , pulling the stem off won't result in the fork falling out.