I noticed that +/- 5mm around the optimum saddle height results in either muscle soreness around the knee (if too low) or in low pedaling efficiency and poor stability (if too high).
This applies for road bike, at higher and longer efforts. For a more casual situation, like mild recreational riding in the park, there is no much difference even at -2 or -3 cm.