Pass the bus on the left. If you've been leap-frogging with it, the "operator" will keep an eye out for you in particular. They already check their mirrors before pulling out. These are professional drivers; they know what to look for.
Also, it is worth pointing out that buses don't just "jump out." Even when they do something sudden, it is much, much slower than a car doing the same thing.
Just try to pass as far to his left as possible. That seems safer to me than to risk hitting a pedestrian. Half the time, they have iPods on and are 90% oblivious to the world around them.