You mentioned that you took the spacers out of the headset. Without the keyed spacer, it's not likely that the headset will stay put.
One possible option is to mill the steer tube a little shorter, taking more off the top end if practical. Another idea, if you can't find a headset with a short-enough stack height, is a Chris King Gripnut headset (or a new fork).