Maybe try eating some food during your ride. You are probably burning up around 4-5 hundred calories an hour and you could just be getting depleted.