RUN, do not walk but run to your nearest LBS and have this corrected. What you are describing is very dangerous and must be corrected properly. The stem must clamp securely to the steerer tube and if yours doesn't it's a serious accident waiting to happen.