Make sure the rear triangle (dropouts) is centered along the frame axis (I understand you've done that already).
Measure both the front and the rear chainline, see if they are matched. My explanation how if you need it: link.
Problem you have could be caused by front chainline being smaller than the rear chainline.

Riding small/small combo causes more mechanical losses due to chain being at a sharper angle, and, since fewer teeth are involved, causes faster drivetrain wear. I understand using that in a racing situation, but wouldn't call it a good practice.