There are minor differences in the overall construction of components as you go up. The higher end components are easier to adjust and will tolerate more adjustments.

For example, brake cable adjustment on Sora is a metal ring with rubber o-ring for grip. When the o-ring cracks (aging), you'll either need a wrench or figure out how to replace. On 105 and its a ridged metal adjuster. Slightly, more difficult to grip, but will last forever.

On 105, the brake centering adjusted is a Philips screw head which is virtually guaranteed to have the head strip out after you made 20-30 adjustments (should take a few years). On Ultegra/Dura-Ace this is an Allen key which will last virtually forever.

Beyond that the shifters require less force, operate more smoothly and have shorter lever pulls as you go up.