Not everyone is afraid of traffic. Some of us just don't like being a hindrance to other people, holding them up, slowing them down. I don't like doing that when I'm driving and I don't like doing it when I'm biking. If there's plenty of room for people to get around me then it's fine, but if there isn't room for them to pass then I feel really bad about making drivers wait for me to get out of their way. Drivers who pass me closely in a narrow lane bother me much less than the ones who stay behind me and follow me at low speed. In such cases I'm not afraid; I'm embarrassed.

Of course, I'm talking about polite drivers here. If a driver acts like a jerk, then I don't at all mind inconveniencing him.