Go Back  Bike Forums > Bike Forums > Classic & Vintage
Reload this Page >

Putting the Magnificent 7 Tubing Test to Bed

Notices
Classic & Vintage This forum is to discuss the many aspects of classic and vintage bicycles, including musclebikes, lightweights, middleweights, hi-wheelers, bone-shakers, safety bikes and much more.

Putting the Magnificent 7 Tubing Test to Bed

Old 11-26-23, 08:13 PM
  #1  
Senior Member
Thread Starter
 
Spoonrobot's Avatar
 
Join Date: Jul 2006
Posts: 3,063
Mentioned: 63 Post(s)
Tagged: 0 Thread(s)
Quoted: 1216 Post(s)
Liked 183 Times in 116 Posts
Putting the Magnificent 7 Tubing Test to Bed

This is probably the most referenced, and most misunderstood magazine article ever posted on the forum. It's constantly referenced both for what it says and does not say.

Here is the text, OCR and transcribed as needed by myself:

The Magnificent 7
The Ultimate Blind Test?

By Alan Coté
Photography By Fran Kuhn

Helmets Bell
Jerseys Santini
Riders: Kevin Burke, Shawn Cronkite, Jim Fryer, Mark Rich, Sugi Sorensen, John Wike, Steve Wright

"Your mission, should you choose to accept it, is to test seven Mondonico bicycles," the boss told me over the phone. "The bikes appear to be virtually identical. Identical size, identical geometry, identical components-even the spoke tension is the same."

I tugged at my shirt collar. The boss cleared his throat. "The only difference is the tubing. What we have here are seven kinds of Columbus tubing, but you won't know which is which."

I paused for barely a moment. "Boss, you know this is my kind of assignment," I said.

"Get on the plane to L.A.," he replied. I hung up the phone and tipped back in my chair, thinking for an instant I should light up a cigarette for the imagery.

"I'll need these," I said out loud to no one in particular, as I tossed my Sidi Tecnos and a pair of Speedplays into a black attaché.


MAGAZINE EDITORS love to rant and rave in road test articles. We say one bicycle is especially stiff, while an-other is soft. This bike has twitchy steer-ing, that one is more stable. This one soaks up road vibrations, that one has a buzzy ride. This one is good for sprinting and criterium racing, that one is better for long road races and century rides. Most of the qualities of how a bike rides, of how it feels on the road, are entirely subjective. Some of a bicycle's attributes, such as steering, can be ana-lyzed—if a bike steers quickly, we can plug some numbers in and calculate trail to confirm our inklings. But other qualities about a bike, such as vibration damping, are entirely subjective. Editors and other alleged bicycle au-thorities often attribute these ride qual-ities to something in particular. We say one bike is stiffer because it uses tubing that incorporates internal helical rein-forcements. Or another bike vibrates on descents due to flexible fork blades or because the wall thickness of the top tube is only .3 millimeters. The companies that manufacture bi-cycles and tubing jump right on the bandwagon. They lead us to believe that the more expensive bikes and types of tubing are better on the road. But why is it that we think a very expensive bike rides better than a less expensive model—are the extra dollars sublimi-nally making conclusions for us? This group of Mondonicos could help clear up some of these issues. Among the seven bikes, there were seven types of Columbus tubing. Tubing decals were omitted from the frames—there was no way to tell which bike used which tub-ing.

Each bike simply had a number, one through seven, on the stem. Only when we cried "uncle" after finishing the road tests did Mondonico's U.S. dis-tributor, Torelli Imports' Bill Semonian, let us know which frame was which. Semonian went to great lengths to en-sure that the only differences between each bike was the tubing. The seven frames used exactly the same design and geometry, including subtleties such as each fork blade being raked on the same mandrel. Antonio Mondonico brazed all of the lugged joints himself, while Mauro Mondonico, Antonio's son, torched the braze-ons in place. The Mondonico shop then had all of the frames painted the same metallic green color. Semonian didn't stop there. He had one mechanic build the frames up with Sachs New Success component groups ("All the groups came from the same shipment," Semonian said, though he wasn't sure why this should matter). The wheels were laced and trued, and a tensiometer was used to ensure uniform spoke twang. All other components—rims, saddles, tires, tubes, bar tape, rim tape—were the same across the board.

It was a brown day when I arrived in the City of Angels. I made my way to the boss' mid-Wilshire office building. As I stood in the elevator, the difficulty of my assignment hit me. Sweat began to bead on my forehead. What if I couldn't feel any difference between the bikes—what would I write about? "You'll need the right vehicle," the boss told me, his hands castling ner-vously. / started thinking about a Jaguar, or maybe a big, black American sedan. "A minivan should do it," he said. I'd look like a Boy Scout troop leader, but seven bikes could fit inside. A few hours later, I pulled a Pontiac Trans Sport into the underground park-ing garage where the Mondonicos were stashed. "Head up to Malibu for riding," he told me. I nodded, put on my trendy, I-wanna-look-like-a-pro Oakley sunglasses and headed into the midcity gridlock.

Road testing a bicycle is usually a straightforward affair. We try to ride a bike under a variety of conditions in or-der to report its full range of abilities. We compare a given bike to a mysteri-ous standard—a standard that's the cul-mination of impressions of every bike we've ever ridden. This road test was a little different. What really mattered was the relative differences between the bikes. If I had a year just to write this article, I could have spent a month or more on each Mondonico. I'd get so tuned in to how one bike rides that moving to another might feel like a big change—or maybe not. But by the time I would ride the seventh bike, my half-year-old impres-sions of the first bike would be a distant memory, and you would be reading this article alongside coverage of the At-lanta Olympics. A short riding blitz of all seven bikes would be the test recipe: Set up a bike with the correct seat height, pump up the tires and start pedaling. Repeat seven times. My ride loop, amidst some of Malibu's most posh, ocean-side homes (and the toolshed that BG staffer Mark Riedy used to call home), was the same each time. Included was a mixture of short, steep hills, corkscrew descents and blind corners covered in fine beach sand.

The Mondonicos were stacked in the van in no particular order. My first ride was on number six. I couldn't help but notice the oversize top tube as I cruised along. "Which Columbus tubesets use an oversize top tube?" I wondered, before trying to put the thought out of my head. I liked the way number six felt. It was spunky when I stomped on the pedals while grinding up steep grades. The bike quickly felt familiar and reassuring—it felt like a fine, handmade bike should. Next in line was number two. It seemed to ride exactly the same as number six. Then I jammed up the same steep hill, and the bottom bracket seemed to flex a little more. I moved on to number three. Not sur-prisingly, it felt the same as the first two bikes. In the hill jam, I decided that it flexed a little more than number six but less than number two. And so on. The Mondonicos did not, in fact, all ride exactly the same. I thought number six was the most rigid. Stiffness aside, it was also my favorite Mondonico—I just liked the way it felt. Number seven seemed a little smoother on rough pavement than the others. I decided that number two was the most flexible, though the overall feel of the bike was still quite good. Then'I started second-guessing myself—did number two feel the most flexible because it was the next bike I rode after my fa-vorite, number six? Four of the bikes numbers one, three, four and five—seemed almost interchangeable. As I looked over my notes, I realized how subtle the differences between the bikes were. The ride variations of these seven Mondonicos were nothing like the contrast between, say, a carbon-fiber Trek and a titanium Litespeed, or an aluminum Cannondale and a steel Eddy Merckx, or even between two bikes built by different manufacturers using the same type of material, or even the same brand and model of tubing.
My work in the field was done. "When you're ready, call this number and ask for Bill," the boss told me. "He's the only one who knows the true identities of the bikes."

It was time for the Big Question. How did my riding impressions of the bikes relate to different tubing models? The bike I liked best, which I also thought was the stiffest, was bike num-ber six—the Neuron frame. The one I thought was the softest was number two—the SLX frame. The one that seemed to absorb vibrations best was number seven—the Thron frame. I could perceive no real difference be-tween numbers one—Cromor; three—Brain; four—Aelle; and five—EL-OS. My affinity for the Neuron frame jives with the tubing's résumé. Neuron uses some very tricky butting—the wall thickness varies such that the outside of the tube is round, but the inside is elliptical. The fancy butting can add stiffness and resulted in a frame that weighed 4 pounds—tied with the EL-OS frame as the lightest in the test by almost 1/2 pound. For the SLX frame to be the softest seemed technically possible. While the rifling inside the down tube and seat tube was billed as the road to rigidity 10 years ago, an oversize tubeset with similarly thick tubing—like Thron—should be stiffer. But I picked Thron as the most shock absorbing. And I lumped Aelle, Cro-mor, Brain and EL-OS together. To be honest, I couldn't feel a difference between an Aelle frame—with straight-gauge tubing and weighing in at 4 pounds 12 ounces—and an EL-OS frame—with double-butted, oversize thin-wall Nivacrom tubing and only 4 pounds of heft.

A conclusion which, if marketing literature is to be believed, doesn't make a whole lot of sense. If the numbers on the bikes were switched around and I were to test each bike again, my guess is that I'd come up with different tubing prefer-ences. I think my ride impressions were essentially random. Does this mean you should pass on the expensive steel frame and buy a cheap one? I don't know. The Aelle frame does weigh % pound more than the EL-OS frame, but the EL-OS frame costs $815 more—close to three times as much as the Aelle frame. I'm reminded of something Richard Sachs - one of the finest custom framebuilders in the country - told me. Sachs said, "When someone is buying a bike from me, they're buying my design choices and my construction skills. I actually think material is the least consequential choice." What do these seven Mondonicos say about bike performance and testing? Perhaps more than anything, they illustrate that the personality of a bike is determined much more by fit, frame geometry and components than by what kind of tubing lies underneath the paint.
The test bicycles


My critique:

The Magnificent 7 Blind Test fails at providing a clear experimental design and thus fails to provide a compelling result. The bicycles were ridden one after the other, by one rider, one time. Then the experiment was over.

1 Cromor
2 SLX
3 Brain
4 Aelle
5 EL-OS
6 Neuron
7 Thron

The bicycles were also ordered in such a way that the differences, if they did exist, would be extremely difficult to determine over one ride, with perceptions from 1-2-3-4-5 or 6 bikes ridden directly before.

This is obvious by his discussion of the first and second bikes ridden. First:


And the second, which he thought the softest


Which is exactly the expected outcome if he was only going to ride these two back to back. Tubing diameters the same, reduce DT and CS butting profile by 1-step with a minor variation to the top tube. Any bike ridden after the Cromor bike would probably have felt extra compliant, less than the SLX which was most likely correctly identified as the most compliant by the subject.

Every additional bike he rode reduced his ability to parse the differences. The next bikes in order:










However, at the end of the day he did an admirable job of picking the bikes in the expected order.

Softest - SLX
Absorb vibrations the best - Thron
Unable to differentiate - Cromor, Aelle, Brain, EL-OS
Stiffest and "best" - Neuron

The three bikes that he picked out specifically all have remarkable features compared to the rest. SLX ridden directly after the only 1-7-1 DT bike and featuring SD tubing and the thinnest CS/SS. Thron with the heaviest collection of tubes. Neuron with the most complex butting profile and design - I suspect this last is fairly close to "vertically compliant, laterally stiff" as a concept.

Other limitations:

This is exclusively Columbus tubing with their chosen profile and butting. Are the differences more pronounced with a different brand of tubing? Other testing indicates this is a possibility.
There was only one rider, who rode each bicycle once.
The Mondonico bicycles were a very specific design, level top tube. It's possible other designs with compact geometry would behave differently as their stiffness profile would be different.
Other blind testing with larger subject sets does not replicate this test.
The experimenter was the same person as the subject. His perception of randomness is in error, if the experimental design allows for such a judgement it is not robust to produce any result at all.

Essentially, the outcome of "the rider couldn't tell them apart" is not supported by the experiment, however, neither is any other result. This experiment, due to it's design does not produce a result at all. It is a ride report with some color segments and photos, not a blind test of any veracity.

Other notable tests that do not replicate the supposed result are an earlier test between Tange Prestige and Columbus SL, where two riders were unanimous in their determination of the differences between tubing that was opposite of their expectations. The thicker, heavier tubing rode better for them

https://www.bgcycles.com/new-page-1
The book says that the reduced wall thickness will make the Prestige tubing less rigid. For the midsection of a 1 1/8” down tube, skim 0.1mm of wall thickness and you lose 16% of your tubes rigidity; skim 0.2mm and you lose 32%. The rigidity loss is favored by many riders for its effect on the way the bike feels on rough roads. Some cyclists bemoan any lost rigidity for climbing, and others say it doesn’t matter. That particular debate will never end.

Of course, the skinnier tubes will reduce the weight a bit. Gordon said that on the finished frames, the weight difference was an insignificant 5oz (about 148g) – one third of a water bottle. Our Tange frame had the same wall thickness as the Columbus SL frame in the seat tube (0.9/0.6mm) and fork blades (0.9mm), and skinnier walls everywhere else. The Columbus down tube was 0.9/0.6/0.9mm, Tange’s was 0.8/0.5/0.8mm (that was Gordon’s request: most Tange Prestige frames are 0.1mm thinner than that). The Columbus top tube was 0.9/0.6/0.9mm: Prestige 0.7/0.4/0.7mm. Chainstays and seatstays were 0.7mm for Columbus and 0.6mm for Prestige.

Tiny differences, five ounces of metal stretched out over six tubes. And when I rode the bikes, I hadn’t been told which was which. And yet, there was a definite difference between the bikes and it took little time to notice. Once when I went on a side by side with Imre Barsy, our former industry editor who now works at Specialized, he took all of 100 yards to notice the differences and voice his preferences for the pink bike.
Barsy’s preference was due to the single most striking difference between the bikes: the pink once seemed to transfer fewer vibrations to the rider. The minute vibrations that result when you roll over a slightly rough asphalt road are a source of fatigue and bother. They were much , much lighter on the pink bike.
-
The pink bike, the one Barsy and I preferred was the Columbus SL bike!
Rene Herse has also performed a more recent blind test with three different bicycles and reach a different conclusion than either the Magnficent 7 Mondonico test, or the Bruce Gordon Tange v Columbus test. This test actually featured three riders, two of which were able to make a determination of frame characteristics, and one who was not.

https://www.renehersecycles.com/what-is-planing/
A few years ago, we did a double-blind test. Jeff Lyon built four frames for us. They were identical, except for minor variations in frame tubing (and hence frame stiffness). The differences were small – just 0.2 mm difference in the wall thicknesses of top and down tubes (Bike 2), or 1/8″ in the diameter (Bike 3). Bike 4 was a duplicate of Bike 1, as an additional check of our results. Apart from those two frame tubes, all four bikes were absolutely identical – same tubes, same geometry, same paint, same components. Even the same weight: The lighter frames had weights inside to equalize the weight.

The goal of this experiment was simple: We wanted to see whether small differences in frame tubing are discernible to the riders, and whether they make a measurable difference in performance. Would our riders prefer the stiffest bike? Or the most flexible? Or would it make no difference at all?
The test was a true double-blind test. Neither test riders nor test administrator knew which frame was made from which tubing. To hide the tubing diameter (one frame used oversized tubing), the bikes were wrapped in foam insulation. In every way, the test met the most rigorous scientific standards.
We rode the bikes in a variety of tests. One of them was an uphill sprint for 340 m (1100 ft), with two testers racing each other. Both bikes were equipped with calibrated power meters. We repeated the sprints five times, with the riders switching bikes after each run. After the fifth run, the riders were exhausted, so we stopped the experiment. It’s one of half a dozen experiments that all showed the same: Small differences in frame tubing can lead to a significantly different feel and performance.
It should be obvious now that one test, featuring one rider, who rode each of 7 bicycles one time is not robust enough to make any determination. The test does provide a framework to think about and approach understanding how to determine differences but it is not a concrete result to refute or confirm conventional wisdom, anecdote, or other types of personal data. What was the result of the Magnificent 7 Blind Test? There was no result, it does not meet the criteria for a blind test, nor an experiment.
Spoonrobot is offline  
Likes For Spoonrobot:
Old 11-26-23, 08:43 PM
  #2  
weapons-grade bolognium
 
thinktubes's Avatar
 
Join Date: Dec 2008
Location: Across the street from Chicago
Posts: 6,344

Bikes: Battaglin Cromor, Ciocc Designer 84, Schwinn Superior 1981

Mentioned: 44 Post(s)
Tagged: 0 Thread(s)
Quoted: 985 Post(s)
Liked 2,377 Times in 891 Posts
Fake, but fun!
thinktubes is offline  
Old 11-26-23, 11:59 PM
  #3  
Senior Member
 
TenGrainBread's Avatar
 
Join Date: Sep 2015
Location: Madison, WI
Posts: 2,701
Mentioned: 52 Post(s)
Tagged: 0 Thread(s)
Quoted: 1136 Post(s)
Liked 650 Times in 336 Posts
If only these magazine authors were required to take a 101 level statistics or design-of-experiments course before writing articles like this.
TenGrainBread is offline  
Likes For TenGrainBread:
Old 11-27-23, 01:01 AM
  #4  
Senior Member
 
Dfrost's Avatar
 
Join Date: May 2013
Location: Pacific Northwest
Posts: 1,989

Bikes: ‘87 Marinoni SLX Sports Tourer, ‘79 Miyata 912 by Gugificazione

Mentioned: 166 Post(s)
Tagged: 0 Thread(s)
Quoted: 500 Post(s)
Liked 466 Times in 256 Posts
Originally Posted by TenGrainBread
If only these magazine authors were required to take a 101 level statistics or design-of-experiments course before writing articles like this.
BQ’s second rider, Mark Vande Kamp, often seen in side-by-side photos with JH, is a professional statistician in his “day job”. I don’t drink all the BQ Kool-Aid, but do feel that their testing is done with transparency, honesty, and statistical accuracy.

Last edited by Dfrost; 11-27-23 at 01:06 AM.
Dfrost is offline  
Likes For Dfrost:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Thread Tools
Search this Thread

Contact Us - Archive - Advertising - Cookie Policy - Privacy Statement - Terms of Service -

Copyright © 2024 MH Sub I, LLC dba Internet Brands. All rights reserved. Use of this site indicates your consent to the Terms of Use.