imo, compact design wasn't totally viable and largely unexplored until after "oversize" and shaped tubing became readily available. with "standard" tube diameters, lateral stability in locating the headtube/steerer was too compromised. oversized top & downtubes created a much more stable steering geometry which could be exploited without serious compromise. it's my personal belief that there's an additional factor to consider regarding longitudinal stiffness and handling/cornering qualities- ie: that any sideways frame deflection exhibited should ideally be even & progressive and occuring along the entire frame's length, from steerer/headtube to the rear dropouts or else a "hinging" behavior will occur where one or more segments of the frame deflect at a different rate or load than the rest of the structure, leading to unbalanced vehicle dynamics, loss of adhesion and unpredictable cornering behaviour.