I also sometimes lower my saddle for various reasons, including fatigue. I've read pros often do the same, even during multi-stage events like Grant Tours. I also quite often experiment with saddle height on my trainer and have found that I can easily tolerate lower than optimum saddle heights much better than higher ones. For example if my preferred saddle height is 780 mm, I can drop it as far down as 750 mm without any real issues, but I would get sore hamstrings if I raised it to 790 mm. So it seems as if there is a much wider tolerance window on the lower side of optimum saddle height. I know a few very experienced fitters who have stated this too.

So for me, riding a couple of miles with my saddle an inch or two below optimum wouldn't bother me in the slightest. In fact for a casual slow ride I would probably prefer it.