Another issue with pulling spacers is that your axle might protrude too far into the dropout, rendering your quick release skewer ineffective (and resulting in your axle slipping under load).