The biggest advantage of the expensive bicycle is less weight by making your wallet lighter. Another advantage is that sometimes other riders will marvel at what a nice bicycle you have.
The expensive bicycle is expensive because people will pay that much for it, plus the cost includes all the research and development, manufacturing, overhead, blah blah blah required by the companies involved plus some profit. There is probably less work overall in putting together the lower priced bicycles because most of the research, development, and testing was probably done on the higher priced top of the line stuff and the lessons learned from there can be trickled down to the lower price point bicycles. As mentioned, you generally get nicer and lighter components as you pay more.
As far as performance gains, in my experience, they are extremely minor if any. If you are getting killed by your riding buddies on a $1500 bike, then you'll still get destroyed on a $10000 bike. If you are racing, then maybe the very small advantages could make a difference over your closest rivals when you're talking about a few seconds here and there.
I have bicycles that span that range, and I enjoy riding all of them. I pick which one to ride mostly based on whether there is a good chance of rain and whether I'll be riding in a group with some lousy bike handlers and/or lousy pack/group riders that will increase the chance of crashing me out. Most of the time I prefer to ride my most expensive bicycle which is also my favorite ride, but under adverse conditions I usually choose the least expensive bike. To be honest though, I'd be perfectly happy if I circumstances were such that I could only have my least expensive bike because it's good enough and still lots of fun to ride.