The solvents are not the problem, but the microscopic metallic shawings in them, and you have them whether you used detergent based or organic solvents. That thin metallic dust contains mostly iron, but also, sadly, chromium and other alloying metals.
That, and the dissolved lubricants - some of them can be somewhat noxious.
BUT not MY chain lubricant - I use a biodegradable chain lube!