Yeah, buses are definitely scary like that because of the silence...same with hybrids in electric mode. It really seems to me like some drivers just need some better education/training on what to do in various situations. What we have in the U.S. right now is woefully inadequate, especially with respect to how they should properly share a lane with a bicyclist or even a motorcyclist or scooter. I'd like to see some testing of actual driving skills, perhaps on a simulator, rather than a cursory "do you know how your car works" checklist. I can't understand why we don't do that now, given the number of deaths we have on our highways every year and given that it would help improve people's driving skills, period, and not just in dealing with bicyclists.
Of course, bicyclists and road engineers probably bear some responsibility in forming this sort of driver behavior. If 90% of the bicyclists that drivers pass are either in bike lanes or gutter bunnies, some of the dumber drivers are probably going to come to have the expectation that you always pass a bike on the left instead of the right depending on the situation. That's still no excuse for ignoring hand signals, but it's something of an explanation.