If the stem is turning as much as it is, it's unlikely that it's not dislodged. And if the stem is broken and being held together by the bolt / wedge nut, then it does need to be removed completely. At least, of the few broken stems I've encountered, I've had to remove the bolt to release not only the wedge, but the broken-off lower bit of the stem.