I've had a few moments where drivers reacted unpredictably, meaning "I thought they saw me, but they didn't." It's always been these cases when I've used the brake.
I go for weeks at a time without using the withered handbrake-- just a supplementary cross lever on the front. I play this game with myself where I really try not to use it unless I really have to. And I know it ain't safe.