Sure that's true when you're talking about difference of 10+ pounds. I was specifically thinking of rides like my 10 mile commute, even including its many stops and rolling hills. Also, while weight (within reason) doesn't make a difference in the time on a trip like that it can make a difference in how tired I am when I get there so for a really long trip with a lot of starts and stops a big difference in bike weight can have an effect.

Even for the small sample of crossing an intersection I don't think wheel weight would make a difference. Though, again, if you're racing the minute difference it makes in a sprint could be the difference between winning and losing.