High-fat food is bad for you whether you ride or not. The people I know who play the "I can eat anything because I rode" game all carry more weight than they want, except for the ones who ride serious amounts of time.

For your situation, I have two thoughts.

1) 250 cal/hour of carbs during your workout will help in keeping your blood suger up, and mute the hunger.

2) A good recovery meal asap after you've finished exercising will help more. I use a recover drink (endurox), and it helps me a lot with this.