Compact frames (i.e. sloping top tube frames) were created to tighten (i.e. make smaller) the frame triangles, particularly the rear, for less flexible, snappier frames.

It was Giant who popularized the compact frame (for industrial manufacturers) in the mid '90s with their TCR. The rethinking of classic road geometry grew out of the cycling milieu created by MTB demands and the rush to create lighter bikes and lighter tube sets, and the fascination with titanium. Lightweight tubes, at that time (i.e. late '80s) meant whippier frames, so MTB builders were forced to rethink traditional road frame construction. When Giant brought the philosophy to the road, roadies sneered because the frames looked like MTBs, but they were, of course, on the wrong side of history in that regard, first because MTBs didn't look like that for long, and the benefits of what came to be called 'compact geometry' were irresistible.

Those benefits, in addition to the driving frame stiffness gains, were weight reduction and production and stock economies. Whereas traditional geometry limited the range of people who could fit a given frame due to standover limitations, compact geo allowed a wider range of people with its lower standover heights. Add to that the rise of the Ahead set and stems and the associated ease of bar height and reach adjustments, one threadless fork could be produced, rather than a slew of threaded ones, and swapping stem lengths to adjust reach became fast and easy.

In other words, going compact was a no-brainer for manufacturers. Going back to Giant, if I remember right, that first TCR was only available in S, M, L, and XL. Most manufacturers up until that time were wrestling with how many of each of 10 or 12 sizing increments to produce and how to distribute them to dealers efficiently.

So, compact was really a boon for both manufacturers and consumers, in the sense that manufacturers gained efficiency, and consumers better performing bikes. Theoretically, consumers should have also gotten better availability of a bike that fits, and I think that's probably true (especially for women); it's common to see stems swapped, flipped, raised and lowered on the showroom floor as buyers go out on test rides now, where in years past, it just didn't happen. Bars up or down, that was about it!