One important factor for going smaller ( my preference generally and I'm a 5'10" guy) is that a smaller frame is always appreciated when your boots sink into soft snow---your crotch won't like a frame that juuuuuuust fits you on hard pavement.

Just today I was riding with a woman friend whose fatbike really should be a small, is a medium, and when you sink in snow or stall out on a climb due to lack of traction, one appreciates a smaller frame.
Today we were on some really hard trails and I sure as hell wouldn't want a larger frame, having to suddenly jump off my bike, stalled on some very steep and narrow trails, with a steep dropoff to one side, which happened to me and I was getting tired too.

My wife is 5ft 1 or less and her small fatbike frame is sized ok, but close. We just couldn't find an XS like her touring bike, which fits really well.
Don't forget though, different frames have different frame shapes, so toptube positions do vary.