Are you asking about deterrents for people who have never commuted by bike, or deterrents for current commuters? I think the answers would differ.
I am a female in my early 30s and I work in a tech job in Northern California. I'm very lucky in that the weather is always good, aside from the rainy months, which aren't even that bad.
Here's a little thing: it's hard to be fashionable at work (styled hair, clothes, sometimes makeup, etc) when I have to drag everything with me in a bag, and wear a helmet, and sweat profusely. There's more pressure on women to spend time on their appearance, and cycle commuting is adversarial to that. It's the little stuff: my skin is flushed most of the morning from the exertion of the ride, for example. No way around it. Sigh. I'll never be the belle of the cube farm. Luckily, I don't have dress business-formally for my job.
Of course, the previous posters' comments about traffic, laws, visibility, and safety, are also annoyances for me. But they said it better than I could have, so I've nothing to add.
And, certainly, none of these annoyances will deter me. The only things that would deter me are 1) if I got injured and physically couldn't ride (god forbid, knock on wood); 2) if I worked at a place where I'd have to get dolled up and they didn't have the facilities to do so; 3) if it was farther than my physical abilities.