are fitters scaring people into thinking it's rocket surgery?
No, it's a result of a few people, on forums like this one, that worship the very ground that bike fitters walk on.
Changing a saddle is a bit trickier than other component that can change the fit of the bike. You can take all the measurements you want, but it will only help to give you a starting point. Saddle shape and padding density will change the position of the body slightly, so some adjustments will likely be needed to get it perfect.