I have the 3-speed bar end shifter for the S3X and it is louder and stiffer to move when it is not installed. You have a lot of leverage when it is in the bar so it is easier to move. I think it isn't actually quieter when it is installed, rather there is enough wind noise and other noise that the sound is not noticeable.

Mine doesn't wobble from side to side at all. If yours does, a nylon washer might be a better solution than a felt washer. The cable tension might get rid of the wobble, though.