This might be a stupid/noob question so I apologize in advance.

I can't seem to keep my rear brakes adjusted properly and it is kind of driving me crazy. I am pretty new at working on bikes but I'm pretty sure there is nothing I'm missing.

The brakes are Dia Compe G 500 with Dia Compe aero levers.

The caliper itself is tight, I line up the pads a few mm from the rim, pull the cable tight and tighten the nut that clamps on to the cable. Brakes beautifully..for about 20 minutes then I check and each pad is half an inch off the braking surface. I have done this 5 times or so now.

I know I am tightening the cable because it is not slipping, could there be something else causing them to keep loosening up. Caliper no good? Or am I just crazy?

Here are a few pictures for reference.






(mind the crappy front wheel/tire, I got a flat yesterday and just threw a spare on there for now)