One gets what one pays for...

Obviously, in some pedal/cleat setups that's acceptable, but it sounds like you do not want the "float"...if the cleat or the pedal is defective, there's not much you can do but return them...but make sure the cleat is attached fully to your shoe.