If your bike measures 23 inches from BB center to the center of the top tube along the seat tube, you have a 58 cm frame. It's entirely reasonable to go down to a 57 cm. But consider: your seatpost will stick a centimeter or so farther out of the seat tube, usually no big deal. But your handlebar stem will also need to stick out another centimeter or so, to keep the saddle-bar relationship as it is on your 23 inch. Not all stems can actually move out another centimeter and still have a safe amount of stem inside the fork tube - that's worth looking at.
What you have now is akin to the so-called French Fit, explained on the Competitive Cyclist website in their notes on fitting a bicycle. Nothing wrong with a French Fit, you may even want to keep it that way after you read the article.
Road Fan
Road Fan