I am rather confused myself. The top cap and the star nut or whatever preload device you are using has one duty, to preload the steerer so that you can then clamp down the stem. The stem itself is what keeps the preload in check. After you preload the steerer with the top cap, you then tighten down the stem firmly as you can for the type of steerer you have, you should then be able to completely remove the preload cap and still ride the bike without it loosening up since it is the stem that keeps the preload proper, not the cap any longer.

If it is loosening up, I might look at the issue in a different way. I imagine that maybe the stem is sliding up the steerer during bumps. Use some sort of gritty carbon paste on the stem/steerer to prevent it from sliding?