Just noticed that the crappy old front brake on my fixed gear bike flexes forward when force is applied, i.e. the rim pushes it forward a lot. It's a longer-pull brake, maybe made for bigger wheels/fenders, but it looks like it's flexing forward quite a bit, maybe an inch, maybe more. Is this in danger of breaking, or is it normal braking ? I tightened the binder bolt on back of the headtube, but that doesn't seem to have done much. What else should I do?

Get some decent braks. Shimano 105 dual pivots are good value and massively stiff. Get the correct drop for your fork.

nothing wrong with old sidepulls. You need to tighten the arms, on old ones there's usually a nut & locknut system on the front adjust it like a headset... Tighten the nut so there's no more play in the arms (or at least at lot less, but not too tight) then tighen the locknut while holding the adjusting nut.