You might find good advice on the Bicycle Mechanics forum.
Meanwhile, is your brake cable pulling or pushing on the caliper because it is too short OR too long, respectively? Is the knurled washer, that keeps the caliper straight, missing? Are you adjusting the caliper position as recommended by the manufacturer, not just "ham fisting" it. Which brake, front or back?