I suppose that's true but it has never happened to me (that I know of). A quick inspection will reveal if you have a problem.

I have picked up grit from dirty roads but you can hear that and pick it out of the pad. It's a maintenance item.

The abrasive on garnet paper is not well bonded to the paper but is designed for the abrasive to break down (crumble) with use.

Last time I used sand paper it was a high quality 50 grit with abrasive that is bonded to the fabric substrate. It made quick, easy work of toeing in the new Campagnolo pads.