Not being a physicist, I'd reckon there'd be some improvement, but cannot say how much. There was an article ~6 mos ago floating around about a UK doctor that recorded that he noted no reasonable change in speed between his cheap steel road bike and his expensive carbon bike (the notion itself torqued off a lot of folks - there's talk that the piece might have been satire). Somewhere here on BF, someone claims that he's faster on his Hybrid than his road bike because he doesn't sweat plowing through potholes and over rough terrain on the hybrid whereas he's concerned about damaging the road bike.

If nothing else, I'd reckon that on a lighter bike you'll be able to get up to speed faster when leaving a light, so that should shave a little time.