Major difference, other than the cost of each group is the quality of manufacture and weight of the components. The good stuff is better built, weighs less, and costs a lot more. Some say that the better stuff lasts longer also. Within each manufacturer is one group that hits the sweet spot in relation to cost/performance. It is usually an upper mid-grade group, for example Ultegra in the Shimano line or Force in the SRAM line.

p.s. Yes, one can tell the difference between groups. Not necessarily between two adjacent groups, for example, (I'll use Shimano groups), between 105 and Ultegra or between Ultegra and Dura-Ace. But, between 105 and Dura-Ace, one can certainly tell a difference, or between Tiagra and Ultegra.