Depends on where you live. In 5 years I've been yelled at 3 times and have never had what I'd consider a scary pass; probably nothing closer than 2 feet and I don't think those were on purpose. I also can leave my bike on the sidewalk in town for hours and it'll still be there when I get back.

Also, I've read in a few places that Canadians can be just as much trouble for cyclists as americans. I'm reading a touring journal right now of a Canadian who rode to the tip of South America and back, he's back up in Canada right now heading home, and he said that in one 30 minute stretch of road in Canada he had more people yelling at him, close passing and honking than he has had in the last 4 years of traveling through the US, Mexico, and South America.