Get one of Rivendell's steel replacements for carbon forks?
Torque wrenches are expensive, but Ritchey makes a TorqueKey, Bontrager makes a similar tool, both are 4mm hex wrenches set at 5Nm, which is where you want to be for steerer and hbar clamps on your stem. Something like $20 for a tool dedicated to not overtightening stuff on carbon bikes and components is way cheap peace of mind...