Sometimes this can depend on pad material, sometimes it can depend on brake adjustment, sometimes it can depend of the brand and build of the brakes.

You were right to try to toe in the pads. However, if your fork blades (or suspension fork) twists enough, or if the cantilever posts deform enough, sometimes the hoped-for effect of toeing in will be nullified.

You didn't mention what brand of brakes and pads you are using, and I imagine that would help to diagnose the problem. Some brakes are more prone to this behavior than others. You also didn't mention if this problem has always been present, or whether it is a recent development.