Largely it's because it's a Paramount. Though some think every bike that had more than one gear should stay that way, no matter the quality.
I don't care what you do, honestly. It's your bike, and it's not stupidly rare. As long as you don't repaint it, grind any braze-on's or hangers off most people will get over it. Keeping to original parts is also a good thing just so if you sell it you may be able to get more from it.
Bottom line, as long as you can turn it back, most will get over themselves. If not, well, it's your bike and they can piss off.
-Gene-