Maybe chewing gum. Seriously, the magnet may like the more massive pieces of metal better. Something sticky on a stick may be able to retrieve the errant part. Another alternative would be to run a loop of thin supple string like fish line down a tube and snare the part, then tighten and retrieve.
Good luck