I've had the same problem. I hadn't thought of the idea that the headtube angle had something to do with it. But I'm guessing I have a similar headtube angle to a lot of CX bikes with my bike, which has a touring frame.
I'm convinced it's usually a combination of having your headset bearings loose and having your rim be worn unevenly. Uneven rim wear (or small dents in the rim) probably makes it more likely for your headset to loosen up, and having a loose headset makes it more likely for your rim to wear unevenly.
What I would suggest you do, if you can, is replace your headset bearings and your front rim. If you have serviceable headset bearings, not a cartridge, you should also consider either replacing the bearing cups or taking them out and putting them back in, in a position rotated 90 degrees from where theys started. (put a mark on them so you can tell). Doing that will place the badly-worn parts of the bearing cups in a position where they won't be in the positions that take the most stress. you can remove your bearing cups using a flat-head screwdriver and a hammer (get the bearing cup partway out on one side and then switch sides as often as is necessary). When you put the bearing cups back in, they'll settle in to position and the bearings will loosen some, so re-tighten them promptly or your brake problems will return.
The advice to use stiff brake arms is also very worthwhile. Another small factor could be wheels being out of true/out of round.