Try adjusting the tiny set screws on each brake arm. They control the return spring tension. That is your problem.
Also, if you installed these brakes yourself, make sure you allowed enough cable housing slack to allow free movement of the arm.

George