Silicone is tough to get rid of since many solvents won't touch it. I'd suggest a thorough scrubbing using plenty of hot water and Dawn dish liquid before trying any aggressive solvents.
The spray might have been something other than silicone, since silicone usually doesn't harm rubber.