Drawing salve.
Drawing salve (a.k.a. black ointment) is available at any pharmacy. It brings a boil to a head and "draws" the infection out of it. Apparently you have a fairly deeply infected hair follicle.
When riding, keep using the Chamois Creme to reduce friction. You might also try putting a Dr. Scholl's corn pad or a bit of moleskin around the site to help keep the friction to a minimum.
If the drawing salve doesn't cure the problem, you may need to see a doctor to have the boil lanced. A doctor can also prescribe an appropriate antibiotic.
Underwear Nazi
And, oh yeah, if you have been wearing undies under your bike shorts, stop it now!