A couple of points to consider...
If you are on a flat road, and riding a constant speed, weight difference is virtually meaningless. The main issues impacting speed are power input, and wind resistance.
If your friend has a bike that weighs 5 pounds less than yours, and he weighs 10 pounds more than you do, he is still pushing 5 pounds more than you are.
As mentioned, a lighter bike will be a small advantage on any climbs or accelerations (including after corners).
A light bike will theoretically be a slight disadvantage on descents. Remember that objects fall at the same rate regardless of weight, but this is in a vacuum... with wind resistance, weight is an advantage.
