What kind of brakes? As a starting point, clean the brake pads, toe them in, and make sure everything is snug. If you have cantilever brakes the problem may be solved by using a fork mounted cable hanger instead of a stem/headset mounted one.