It could be crud in the cable housing.

I've had sweat evaporates accumulate in the cable housing but they tend to render the brake totally inoperative.

Or you could have a burr on the cable housing at the entrance or the exit that's restricting the free movement of your cable.

If you were to remove the wheel and the cable from the brake, when you squeeze the calipers together are they "pushing" back against you and snapping back into position?

Did you lube the pivots well with Tri-Flo or a silicon lube?