I have this problem every now and then on my commuter bike... seems like especially when I wear light-colored slacks (short commuting sections, so work clothes are fine).

After washing and drying they ay be hard to get out... best to treat them beforehand. Here's how I do it:
Lay the fabric out flat on a smooth surface like the top of a washing machine. Apply some "Goop" hand cleaner (no pumice) and rub it in. Take your thumb nail or a suitable dull, hard plastic object and rub hard over the stain. Add more Goop as necessary. Rinse with hot water to remove the Goop, then throw in the wash. Sometimes I will pre-treat the stain with laundry detergent after the Goop treatment, but the Goop is pretty effective. It does not always work, of course.
Steve