To make an intelligent reply, you need to post your overall height and an accuarate cycling inseam (floor to saddle-like crotch contact in bare feet). You also need to post the make, model and exact size of frame that you have.
As a starting point for saddle height, I always suggest setting the height so you foot is horizontal with the leg fully extended, at the bottom of the pedal stroke. You should be searing you cycling shoes and be clipped in. With that height, it require a 2-3cm rise of the heel to create the recommended 30 degree bend in the leg at the bottom of the stroke, during normal pedaling.
The difference in the fit between a 54 and 56cm of the same model will usually be only 10mm in stem length and 20mm in bar height, so the smaller frame requires a longer stem and either more spacer under the stem or more stem angle, to produce the same bar height.
Someone who thinks that a 5-7cm drop from the saddle to the bars is "normal" might think that my frame is too small, but I use an 11cm drop. I get that with one 5mm spacer under the stem and the lowest stem ongle of 73 degrees. I use a very normal 110mm stem length on my 51cm frame.