In case anyone is still interested
I took a link out but made sure that overall the chain tension was looser. I can feel some slack in the chain when I pedal, whereas before it was pretty uniformly tight. I haven't had problems with the chain coming off, but I had a new creaking sound from the drivetrain I didn't have before. Tightening the cranks and pedals didn't help, and I ended up taking the crank off and greasing the pressed services which the Park blue book suggested. That didn't help at all, but as I was putting the crank back on I noticed that the chainring was extremely loose - basically only 1 screw was still tight, the others were all loose. I tightened those up which took care of the creaking, and now I am wondering if maybe the loose chainring was exacerbating the problem with the chainline, resulting in thrown chains.

As a side question, does anyone think that braking on a fixed gear, i.e. by resisiting the pedals and especially skid stops, tends to loosen up cranks more quickly than road bikes? And maybe chainrings as well?