As long as the metal is dry and doesn't have a wet or greasy feel, you can rest easy that there is no residue to worry about.

One other place to check is inside the hollow tube in the center of the hub that the axle passes through. Be sure there is no water or dirt left there. Pass a small piece of rag through it with a dowel or screwdriver blade if it needs to be cleaned out.