A few years ago, I dug into this because I didn't understand why bike manufacturers had tried to fix what wasn't broken.

As Raoul Luescher points out in one of his videos, an important thing to understand is that the threads in a threaded bottom bracket are a hold-over from the days of cup and cone BBs. Threads are extremely uncommon in bearing seats outside of bike BBs. Subsequent solutions (like cartridge and Hollowtech 2) were made to work with the prevailing threaded standards as that's what most bike frame were using, not because that was necessarily the best choice from an engineering perspective. Threadless came about in the mid-late 80s as an engineering-first solution from small manufacturers who had the freedom to break away from BSA as they weren't producing at scale.

Given that context, here's my understanding:
1. In the very early days (Klein, Merlin etc designs), they were trying to reduce threaded creaking (see my post above), allow the use of sealed bearings, simplify things and offer better bearing alignment.

2. Cannondale acquired Magic Motorcycle and used their BB30 design in an attempt to:
a. make BBs lighter by eliminating redundant material in the cups and using aluminum spindles
b. make the BB stiffer by increasing the spindle diameter
c. make the BB more robust by increasing the bearing size
d. introduce a significantly larger BB shell which offered larger attachment surfaces for Cannondale's fatter frame tubes.
e. Cannondale was trying (note: trying) to reduce manufacturing costs by simply line boring BB shells. Accurately threading a BB shell is not a trivial matter.

4. PF30 was introduced by Cannondale for two reasons:
a. PF30 uses squashable Delrin cups which theoretically allow the system to be more tolerant of bad bearing alignment.
b. use carbon cups and make things cheaper to manufacture. Bonding a aluminum shell into a carbon frame is tricky to do accurately and introduces the risk of galvanic corrosion
... IMO, introducing PF30 is where things really went off the rails as it introduced all kinds of other problems including allowing even worse tolerances, quick wear, narrow bearing spacing etc.

5. Shimano worked with Giant to develop PF86 as they wanted to continue using the wider bearing spacing and 24mm steel spindle from Hollowtech 2. It also is more tolerant of poor bearing alignment as there's more squashable Delrin in there.

6. Trek introduced BB90, FSA introduced BB386Evo, Cervelo introduced BBright, Colnago introduced ThreadFit (etc) all trying to solve the various problems of BB30 and PF30 while trying to avoid patent infringements.