Make certain that your brake cable housing ends are properly prepared (ground of filed flat) and fitted with the correct metal ferrules where needed. Unless the adjuster is almosr coming out of the brake I think you needn't worry about them failing. There is nothing wrong with backing off the adjustment a few turns when you need to open the brakes and then readjusting it when you put things back together.

Do the brakes stop satisfactorily? Having a hard lever feel does not equate to having good braking, and adjusting them tighter to the rim doesn't change the feel significantly but only makes them grab sooner in the lever travel. You might also consider changing to a harder brake pad compound. You might also be feeling flex in the brake arms themselves or of the fork.