Exercise is where we break ourselves down. Rest/recovery is when we rebuild ourselves stronger.
It sounds like you are doing a very intense workout. I find that the day after I ride hard (when the endorphins wear off) I feel pretty horrible. Try dialing back the perceived exertion a notch to see if you're still as wrecked.
Consider ramping up your oatmeal/coffee routine with more protein. I mix in a 6-oz carton of low-fat, no-sugar added vanilla yogurt into mine and sprinkle sliced almonds atop. I add skim milk to my coffee (lattes).
Make sure you get a decent warmdown even if it's some brisk walking after you get off the bike; and don't settle down too long if you have a desk job. Warm showers in the evening, not long hot ones that dehydrate you.
Plenty of water through the day.
There are plenty of other little stressors - getting chased by dogs, sunburn, near-misses by bad drivers, - these too can leave you pretty tense.