It largely a physics problem. The less weight that a vehicle carries, the less energy is required to move it done the road. Or, conversely, the less weight a vehicle carries, the same amount of energy is going to propel it done the road faster...to a point. You do have to take all the forces that work against the bike and rider and those forces tend to increase exponentially with speed but, yes, weight does make a difference.

That said, there are trade-offs. The weight of the system is important to consider. For your example, you have to consider the weight of the rider (doesn't change all that much), the weight of the bike (can vary quite a bit) and the weight of 'other'. If you are going to have to carry 10 kg of other stuff, dropping a few kg from the bike probably isn't going to make that much difference. It may even cause other problems. 10kg of unsuspended weight (the rider has its own suspension) on a very lightweight bike is going to make the frame do things its not made to do, like flex and twist under load. A heavier frame is usually built for dealing with a heavier load so it doesn't flex and twist as much. Flexing and twisting can take away from the energy you have to propel yourself down the road.

You have the perfect test bed to do your own experiments. Take the bike that you currently are riding with all the stuff you ride with. Ride it over a course of known distance that you lay out. To do it properly, ride two different course. Do one in an urban setting with lots of stops and starts and one on an open road. Time the ride and then repeat it without the load. To control the variables as much a possible, do it at the same time of day under similar weather conditions. Do only one ride per day so that your energy levels are the same. Pump your tires to the same pressure before each ride too.

If you want to go way over the top(from 4 rides to 12), you could do the rides in triplicate so that you take the variables like wind, weather, rider condition, etc into account. Further randomize the runs so that you don't do all the loaded urban runs, then all the loaded rural runs, and so on.

If you wanted to go way, way over the top, add in a second bike that is significantly lighter than the original bike but has the same rider configuration. Now you are up to 18 rides.

I predict that you will find that weight makes a difference. What you will probably be amazed at is that the difference in speed is small. A light bike is faster but it's not stupendously faster. You'll probably gain 1 to 5 kilometers per hours but not a whole lot more. You can spend a whole lot of money to get a bike that is a whole lot lighter but you only go a little faster. The returns diminish pretty quickly. If you are competing at a very high level, the investment can be worth the returns. If you are an average joe, your return on investment is much lower.