The bolt from the top cap to the expanding plug (or star nut, in steel steerers) provides the preload for the headset bearings. If you remove it, your fork will fall out. Crashing, which is what tends to happen when the headset comes apart, tends not to be worth the 14 grams the top cap and bolt weigh. (And that's a super cheap aheadset, not one that cares much about weight.)