Very true - go drive the same route that you ride and see it from a driver's perspective. Being in a car just doesn't give you the same sense of space as being on a bike. One look at a typical parking lot will tell you many people have no sense for the physical space their car requires.

One of my usual commute routes has a great shoulder most of the way, which I use as much as possible - so I'm frequently two feet to the right of the white line. The travel lanes are large enough for a car to pass me while staying in their own lane, and still have four or five feet between the car and the white line -- huge amounts of space. And yet they still move way over into the oncoming lane. If you drive the route, you'll see what the driver sees and it's not the same as what we see riding.

Not that it excuses a driver from the responsibility of making safe passes. People don't want to slow down for anything.