So I have an 18650 battery here that seems to have found its way into the washing machine. It has a little bit of brownish staining on one end, where the paper sits around the little bump thing (that's the technical term for it).

Anyway, are these things waterproof?? Is it gonna explode if I charge it or use it in my lights??

No, not waterproof. I'd advise you to discard it by recycling it.

Measure its voltage. If under 3.0v, dispose of responsibly with your local electronics recycler.

Look at the small vent holes on top of battery.. Water easily will get in.. +1 on what is said above.. Cells are too cheap to risk a problem..