First, you may want to check with your doctor.

Your body can deplete stores of easily available energy even if you are not riding hard. You could also be dealing with dehydration if you are one of the many that don't drink enough water, especially if it is hot in your area.

You shouldn't have to either eat or drink within 45 minutes unless something else is at play, but it may be a good habit so that you can take longer rides when you want to.