I occasionally ride all out, and I do love the feeling of going fast (having always been the slow kid) but I'm definitely more of a scenery-checking, cow-mooing, bird-watching type rider. I do really "hammer" it when I'm on the way home and I know I have ice cream in the freezer.
I've found that if I make my rides more about going out and doing and seeing things I tend to go on more of them. Since I started commuting by bicycle (I haven't driven since early April) I've lost 9 lbs without paying attention or trying, where previously no matter how hard I trained I stayed at the same weight (or gained).