As I understand it, the position of the valve stem hole is always positioned directly across from the weld/seam. Because there is more material at the seam, due to pins and/or welds, this allows the wheel to be better balanced due to the additional mass of the valve stem when the tube and tire are installed.
I don't think anyone is arguing that a rim can't be better balanced by filing the seam and/or strategically adding weight. I think most of the "pack" believes it's a total waste of time and/or doubt that you can actually tell a difference in ride quality that isn't due to the placebo effect.
If you can feel the difference in ride between before rim shaving and after, then you'd be able to tell (by ride alone) if a tube has been patched, or even whether or not a valve stem cap has been replaced after pumping up a tire. Not likely.