I tend to agree with this thinking. It sounds like it might be a not badly sized frame IF you were able and comfy about getting into the correct racer's crouch. A 4 inch drop between the saddle and the top of the bars is not at all unusual for a bike being ridden in a proper racing crouch. If you're not comfy riding it that way then you most certainly do have the wrong size of frame AND the wrong style of frame. It sounds like you'd be happier with more of a touring style or cyclocross style setup where the rider's cockpit setup was more upright. Or at least a bigger road frame that lifts the bars up higher. But then that would open up the top tube as well.

Another option, depending on what you have now, would be to switch the stem for one which has a higher angle and keep the fork you have now instead of ending up with a huge stack of spacers. Either way the bike is still going to look odd with the high mounted saddle and bars though.