Blind or double blind tests related to bicycles are intriguing, mostly for their rarity and counterintuitive outcomes, at times.
Jan Heine has one of the few other double blind tubing tests I've found. The full article is more in depth but the blog post has the relevant info:
https://www.renehersecycles.com/what-is-planing/
A few years ago, we did a double-blind test. Jeff Lyon built four frames for us. They were identical, except for minor variations in frame tubing (and hence frame stiffness). The differences were small – just 0.2 mm difference in the wall thicknesses of top and down tubes (Bike 2), or 1/8″ in the diameter (Bike 3). Bike 4 was a duplicate of Bike 1, as an additional check of our results. Apart from those two frame tubes, all four bikes were absolutely identical – same tubes, same geometry, same paint, same components. Even the same weight: The lighter frames had weights inside to equalize the weight.
The goal of this experiment was simple: We wanted to see whether small differences in frame tubing are discernible to the riders, and whether they make a measurable difference in performance. Would our riders prefer the stiffest bike? Or the most flexible? Or would it make no difference at all?
The test was a true double-blind test. Neither test riders nor test administrator knew which frame was made from which tubing. To hide the tubing diameter (one frame used oversized tubing), the bikes were wrapped in foam insulation. In every way, the test met the most rigorous scientific standards.
We rode the bikes in a variety of tests. One of them was an uphill sprint for 340 m (1100 ft), with two testers racing each other. Both bikes were equipped with calibrated power meters. We repeated the sprints five times, with the riders switching bikes after each run. After the fifth run, the riders were exhausted, so we stopped the experiment. It’s one of half a dozen experiments that all showed the same: Small differences in frame tubing can lead to a significantly different feel and performance.
The Magnificent 7 test is most interesting due to it's size and structure. Had the test used statistical experimental methods to construct trial(s) in a different way it would have been possible to achieve a more concrete process-outcome.
As Alan Coté stated, he felt his decisions were essentially random; which was due to the essentially random experimental design.
A more outcome focused design for the experiment would be to bracket the bikes against each other matched to the largest expected marginal differences.
Round 1
Aelle v Cromor
Determine ride feel, "winner" advance
Brain v EL-OS
Determine ride feel, "winner" advance
Thorn v SLX
Determine ride feel, "winner" advance
Neuron - BYE
So then round 2 would be
Aelle v SLX
Determine ride feel, "winner" advance
Brain v Neuron
Determine ride feel, "winner" advance
round 3
SLX v Neuron
So the outcome presents the differences in a meaningful way. For trial 1 of 12 rides of the 7 different bikes. And then of course, the experimenter could take these results and determine the structure of trial 2 to develop a theory of the choices "regressing to the mean" and if they are or were becoming truly random. Other riders could be run through the original and subsequent trial brackets as well to determine a consensus or individual preference weight.
This is not to denigrate Coté but to illustrate how the experimental design constrained his outcome to be less useful than it otherwise could have been.