One of my bikes has rim brakes and a Deore/Acera mix of derailleurs, and the other bike has hydraulic disc brakes and Altus derailleurs. When I notice the biggest difference in the brakes is at the bottom of steep hills, and it is a huge difference in stopping power. As for the derailleurs, they certainly feel different all the time. But the only time I notice a really huge difference is on hills when I need to shift more quickly (especially going uphill) – that’s where the better derailleurs are most noticeable.
So I would say that if you plan to ride a lot of hills, then the better brakes and shifters would be worth considering. That’s a huge jump in price, but it may very well be worth it in hilly conditions.