I have worked on my own bikes for a long time, and I've never used a torque wrench. Nothing has ever cracked or fallen off. :-)

But then I've never owned carbon. Just steel and aluminum.

If I ever did buy a carbon bike, I'd definitely get a torque wrench.