Short ride (<3 hours): when you're hungry / feeling low on energy.
Long Ride: intake about 15-250 calories / hour (you have to figure out the right amount for *you* by trial and error), starting at around 1:15-1:45 into the ride.
That said, there's no real rule, it will depend on what you ate before the ride, how long ago you ate it, what you did yesterday and the last few days before that, how good your body is at switching from carb-burning to fat-burning, etc. etc.
If you are counting calories to lose weight, you still don't want to restrict calories too much during your ride - before & after the ride are the times to eat less.