I agree with everyone on here. The heavy bike will start out slower, but once you get it going, you could go as fast as someone lighter. But there can be a huge difference in tires as well. If you're rolling on knobbies, it'll take more effort to maintain speed. I'm sure going up hills you'll feel a difference, how much? IDK I guess it depends on you. I say that if you feel good about the bike you're riding, then perhaps that will translate into you pedaling harder and going faster. I know everytime I change something on my bike, I think, WOW, my bike is so much better and I ride faster....it's all psychological I'm sure, LOL.
I swapped out my stock boat anchor tires to some Scwhalbe Marathon Supreme in the same size and for me, it seems like I can maintain speed easier as well as accelerate quicker. I think that small drops in weight and you won't notice a difference. I put my bike on a diet and it lost close to 4 lbs. Could I tell a difference? On climbs it seems like it's easier, but I can't tell if that's because I'm getting in better shape or if it was the bike. Perhaps both?