I often rode trails paved with crushed limestone even when they were wet. The powder was very sticky and abrasive. Ordinary cartridge BBs did well for me.

I think you may need to repack more often. The grease itself acts as a seal until it circulates enough to let the grit get to the bearings. Just use a thick grade of boat trailer grease.