I emailed Fizik to ask them about this and see if they have any solution to it and this is what I got:

"Unfortunately not much, leather weeps the color that it is dyed. There are some commercially available waterproof sprays for leather that will help available from any shoe or leather products store."

I don't understand why they would use this leather if they know it's going to be coming off in the rain and onto the shorts. Looks like I'll be throwing this one up on ebay for the ones who actually wear black shorts.