The only thing that I have found that does not scare the people but the hear and notice with out panic is a "little childs bicycle bell" a ding-a-ling.
That is what I use, and I almost always get a "thank you." Fortunately, I only see about 3 people per mile or less on most of my rides, and folks around here are pretty "bike savvy."