Chains with reusable connectors are very easy to clean, you can do both at the same time if you wish. Use mineral spirits or naphtha in an old coffee can or something similar. Start with a long soak, then agitate a bit. Alternate soaking and agitation, until you thing they're clean. Remove the chains, pour off the solvent, and swirl the can to get all the dirt out, then rinse with fresh solvent. You might need 2 or three rinses to make sure it's 100% clean. Then allow to dry for 12-24 hours before relubing with a good chain lube.

BTW- the used solvent should be kept in covered glass jars. The dirt will settle and the clean solvent can be poured off the top next time around.

One other point, Your problem might be in the crank bearing or in the pedals or between the pedal and cleats, so get on your chainless bikes, and work the cranks balancing one foot against the other to create some load and see if the noise is still there. If so, continue to try to isolate, maybe by trying other pedals.