Witterings, keep in mind that in professional fitters terms, "fit" means a true fitting to find the best body position on a bike to start with and is very involved usually cost 150+ dollars. Many, many shops "fit" bikes, but are really only sizing the person to an existing bicycle. There is a big difference. One starts with body position to match a bike to, the other starts with a bike to figure out how to make it work for the body.
In summary, to have a fit done, it will cost you plenty. To have a "fit" done at no cost to you is simply having you swing a leg over the bike, ride it around and have the sales guy observe the body on the bike and make the call. Not the same method or outcome.

With that being said, myself and plenty of other fitters that have been trained in the subject and have fit many, many people, are able to observe and come close with a simple observation due to experience. Won't be perfect nor optimized, but will be close enough to present comfort on most rides.

With that having been said, spend the money and have a pro level fit done and then there will be no questioning. If you are unsure of the competency of the shop you are dealing with, find a different one and work with them.

Now I have to throw a wrench into things. Smaller frames mean shorter top tubes, but also mean a greater disparity between saddle height and handlebar height. Some of that difference can be made up with a riser stem, but if the difference is too great, then there is no where to go. A friend of mine has long legs and short torso and arms. Always had to ride a smaller frame to reach the bars, but the saddle to h-bar height difference always forced him into an unwanted position. After forty years he finally had a pro level fit done and a custom bike made. It has the reach of a medium and a stack of an XL. Kind of an odd looking bike, but he can sit on it for hours, which he was unable to do before.