Really? With only a 1/4" spacer moved from the bottom to the top, it will really 'screw up' the fit. Well, I would certainly try it and if it did not work, reinstall the spacer. Hell, I would take them all out and ride if for awhile. Certainly would not screw you up for life. You could always bring the 4 and 5mm allen wrenches and swap them on the ride.

As for the cable lengths, if you are going lower, it probably not going to cause much difference, moving the stem higher, may, depending on the original set up.

Common, 1/4" is almost nothing.