Brake pads are exposed to all sorts of foreign debris and liquids in normal usage and I believe they ar fairly impermeable. WD-40 is just oil and kerosene. I don't think brake cleaner is necessary, certainly not enough so to get some, and no need for mild detergent on the other end. if you don't already have it. I would tend to use a strong antil grease cleaner, such as 409, Fantastic, etc. Clean the rims with the same thing - won't harm the tire or brake rubber. If you have any problems after doing so then replace the pads.