This is a known problem for certain Oakleys, especially if used in hot and humid weather.
In hot regions of Asia, they know this. I have 2 pairs of Oakleys all have peeled in exactly the same manner, used for about 3-4 years too.
It's not any washing chemical or sweat. In hot regions of Asia, we know better not to use Oakleys, but use Rudy's.