OK, I"m 5' 8" and have been 220 yet nobody would say I was fat at that weight. I was a serious weightlifter. Then I went sailing and needed a different sort of body so I went down to about 195 - again not much fat. Now I"m back to starting on an endurance theme so my weight is even lower.
My point is that you shouldn't care about weight but instead about body composition. The *#&#& doctors and their *#&*#& scales have gotten the world all in a 'weight' conscious mode which is silly. Nobody CARES what they weigh really. The care what their body composition is and what they look like.
If you are serious here, I suggest you get your body composition checked and checked monthly. That will tell you your real progress which probably is better than the scale is telling you. After all, if you lost 50 lbs of fat and gained 50 lbs of muscle, would you complain?