Depends on the OCD level of the rider.
When I ran Tubeless I simply topped up until something else prompted tire removal.
Then I’d scoop out what was easily removable.
Other do remove the gunk - try a net search for ”stanimals” - on regular interval.
I wouldn’t worry about the added weight. IMO it’d take years of top-ups before the weight became important.
Most of sealant by weight is liquid. Which is lost. Which forces the top-up. The amount of particles is low.