When driving a car I'd rather err on the side of caution than allow any desire to pass in a spot that might be too narrow to do so safely result in injury. If I can't see enough road ahead to know I won't be encroaching on unknown oncoming traffic, or if I'm going to turn just ahead, or if the road is too narrow, I'll just wait rather than trying to squeeze by. Yesterday I paced a guy riding pretty slowly for about 3 blocks because of the signal timing, varying lane widths and a need to turn right at the end of the 3rd block. Don't feel self conscious and be happy when a person is more thoughtful or cautious than is strictly necessary. It's certainly preferable to the alternative!