OP, I did it in two stages.In my mid-forties I quit my very remunerative, high-status high-stress job because I wasn't enjoying it any more. It felt as if I existed to serve the job, not the other way round. So I took something at about half the salary that still had enough interest to keep my brain alive, and discovered, I'm glad to say, that being terribly important wasn't terribly important.
Then a year ago, when I was 56, the organisation I worked for closed down. My pensions don't kick in until I'm sixty, and I've always pretty much lived up to my income so I didn't have a fortune. However, I was and am debt free, no mortgage, no dependents. So I decided not to look for another job, and live on my savings until my retirement income materialised. I sold the car (I live in a city) and on the rare occasions when I really need one, I hire it. The only major change to my lifestyle is that I used to spend a lot of money in restaurants, and now use them only rarely.
It's the best decision I ever made. One doesn't need to spend a great deal to live well. I get out on the bike five or six days a week, I'm finally fit enough to race (at least, to race with the Cat4s), I volunteer one day a week at a local cycling-related charity and am on their board of trustees. My favourite vacation is cycle touring, I celebrated quitting work with an eight-week tour, and it's pretty economical if one camps, uses hostels etc. most of the time. And since I can do what I like, I am spending next winter in Southern Spain. Off season, apartments can be rented for a pittance and I can live as cheaply there as at home.
It feels like freedom. If you can, do it.