I find there is a significant difference in the feel of low end to mid grade components. The jump from Sora to Tiagra is noticeable. Tiagra to 105, a little less. 105 to Ultegra to D/A, negligible.

Durability with some components is an issue. Low end components use more plastic parts where the higher end use metal for the same bits. Rear Derailleurs at the high end use jockey wheels with good bearings and tighter tolerances, as well as lighter and often less (of a stronger) material to save weight. At the low end, these parts may be noisy and heavy while still performing their function adequately.

That said, I use a mix of Deore, Tiagra, and FSA mid-grade equipment on my brevet bike. I have old (c. 1991) 105sc equipment on my fixed gear, and I use a 1988 GS200 group and random ultra-cheap parts on my cruiser.