I think I actually do have enough brake adjustability. However, my brakes seem a little screwy or are just different from modern brakes. I remember having some adjustment screws on my mountain bike but on these it seems like the only way to adjust them is to add or remove length from the cable that pulls both cables at once (one cable goes from the right brake lever to a Y cable bracket). Whoever set them up made this pretty short and I'll probably have to redo it and set them a little wider apart as I'm getting an occasional brake rub if I have to hit the brakes hard (shifts the brake assembly just ever so slightly).

If I get free or cheap rims I might consider the swap but otherwise I'll stick with what I have. These rims might have to be trued as they wobble a little bit more than I saw recommended (2mm is what I've read, sound right?) and it might be cheaper to just get new(er) used rims.