Today I rode just 25 blocks for an errand, and there were at least 3 times when cars or buses passed me extremely closely, in not very slow speed (that's the scaring part). One car nearly touched me, but before I knew it, it was already in front of me. It was a very congested street due to constructions ahead. Even though none of the passing vehicles touched me, they were so close that if I had veered a tiny bit to the left, I would have been hit. I am most concerned about being close-passed by buses, trucks or other long vehicles, because a slight tilt of either the vehicle or myself could cause my being hit by the rear part of the vehicle.
In future, I guess I'd better walk the bike on the sidewalk if the street is so congested.