Shimano components are heavier than Sram and some saddles, seat posts, bars and stems can be heavy like boat anchors. And then CAAD10's are something like 100g lighter than CAAD9's.
It's not just one component that makes the difference. One bit here, another bit there and you end up spending another thousand or two to make it just light or stupid light. I think you get the idea.

My bike used to be 18.6 lbs and now it is 17.4 lbs but I'm not faster at all. I also need to upgrade my legs, my heart, my lungs and my fat %.
If you have the money to spend then spend it and enjoy. If you don't have the money to spend then spend your time building a better body. Your bike is fine and looking great.