A big part of why I commute is for my own personal health. If I just take it easy every day then I don't get as much out of it. Now, if I didn't have a 6 month old at home I would probably be less concerned with getting every ounce of a workout from my commute, but seeing as how I do have a 6 month old at home I need to really beat my legs up when I ride.
I don't do it every day...some days I chill and ride at a nice pace and enjoy the scenery.
It's just riding a bike, dude. I'm not out to set any world speed records. And hey, if you're just trying to get places then by all means there's no reason to hammer.