This is true for all situations, from casual road riding to racing on the track. It's on the rider passing to make sure that there is ample room, safest for the rider being passed to do nothing. There is certanly ample evidence that "on your left" is mistaken for "move left" far too often. If your are a fast rider in mixed company you just need to cool it when necessary. Give ample room, Slow down if you have to and quietly ride past.