I went down to about 1500 calories (some days less) but I made sure to eat a balanced diet with plenty of nutrition and no artificial ingredients at all. I didn't feel bad, and I was riding 350+ miles a month. I was hungry, but not starving.
Your body has plenty of calories in reserve, it just needs nutrients. If you're getting the vitamins, fiber, enzymes and enough protein, you don't really need many calories at all. Talk to your doctor, do your homework, and see what works. An all-natural/organic balanced diet did the trick for me.