Well if the seatpost clamp is strong enough to hold the seatpost in place with someone on it it's very like strong enough to hold the headset in place. The quill is secure in the steerer? Where can it go?
You gotta to do it and see does it work. Don't test on the bumpiest steepest hill you can find.
Will it look to goofy or not really be an improvement over a threadless stem, who knows.