Originally Posted by
fstshrk
I think my problem is that my saddle is 3mm too high for the left leg, so if I shim it, it may end up being just right

I get that, I suspect I have something similar, the issue is that the effective leg length is correct currently at the bottom of your pedal stroke but not elsewhere. If I'm understanding your situation you might have a longer left femur, but that is compensated by a shorter tibia/fibula - as if your knee joint is 5mm lower on the left. Does that make sense?