Do you eat something right AFTER your rides? According to the late Ed Burke, it's important to get both carbs and proteins on board shortly after any hard training. The carbs help restore your glyocogen reserves, and the protein helps with muscle recovery/rebuilding.
Some folks use Endurox R4 after rides, but I think it tastes awful (and, it's bloody expensive), so I usually mix up some vanilla-flavored protein powder with 12 oz of orange juice instead.