Wherever you feel confident for the speed, surface and your level of experience. Trust your judgement and hunches. Dont go faster than your abilities for safety sake. There are always the unforeseen gremlins that can jump out in front of you
However, dont let fear hold you back from a good descent if you are continually pressing for a better personal best on a local TT run.
(I'm assuming you are progressing in your riding in the drops as much as possible)