Lighter bike does make difference but does it matter for you? If you want ride fast and race others then get a lighter bike. I prefer slower pace so I can look around and always like to carry some stuff with me and I don't care if others pass me on faster bikes. Fast ride is not fun for me as I need to focus more on the road ahead and have less opportunity to check things out around me. It's not about the ego either, it's about enjoying the ride.

Adam