I for one don't push it on my commute. When I switched from a MTB to a road bike (steel frame) I was faster but some of that has to do with the gearing. If I was pushing for the fastest time I'd look for the lightest bike and not carry anything. There are days were I'm sure the weight I'm carrying is five pounds more than other days and I don't notice that. As mentioned in other posts weather, sleep, and traffic lights make a bigger difference. If I hit the wrong traffic light "it seems like" two minutes waiting that seems to be similar to what some of the other posters have mentioned as gained or lost.
__________________
"I've been dropped a lot of times, but it's never been because of my bike." DXchulo