Correct. The spacers really only position the pads closer to the rim*. Though some will argue that the angle is most important. I suggest you position the pads so they are going to strike the rim, when you apply the brakes, as square-on as possible so the largest area of the pad contacts the rim as possible. Remember to toe-in the pads with the spacers - and test ride to make sure the brakes stop you as effectively as possible.
* and angle the pads with a gentle toe-in position.