Your approach is where I would start. Also, I ride about 1/2" lower than the bike fit calculators tend to recommend. Wouldn't go lower, but might still raise it slightly. Still working on it.

I had a funny experience at a shop... they tried to start with the seat set using my iliac crest (top of hip bone). It was a good 2" too high. I could eyeball it as wrong, so I had them lower the seat, then lower it again after trying it. On my test ride it was still kind of high, but at least I could get some sense of the bike. The other LBS did better.