Jan Heine, Editor, Bicycle Quarterly November 28, 2014 at 7:22 am
#
Just like you, I have published extensively in peer-reviewed journals in my previous career as a geologist.
BQ contributor Mark Vande Kamp still works as a scientist designing exactly these kinds of studies. He frequently publishes in peer-reviewed journals.
Many people assume that
Bicycle Quarterly‘s articles are not peer-reviewed. This is incorrect: All our important test articles are peer-reviewed. This is unique among cycling publications. The double-blind test was reviewed by Jim Papadopoulos and Hank Folsom. Jim is probably the most scientific of bicycle researchers, having published in Science and other prestigious journals. Hank has more experience with frame tubing than almost anybody.
You concern about the study cohort is an interesting one. You misunderstand the hypothesis. If our hypothesis was that “planing” was a factor that influenced bicycle performance for average riders, then our study would have been flawed. However, that was not our hypothesis.
Our hypothesis was that the sensations of accessibility of a bike’s performance that
Bicycle Quarterly‘s testers feel on the road are real and intrinsic to the frame. Basically, we wanted to know how reliable our observations were. We found that they held up in a double-blind test. We could reliably detect very small differences in frame tubing.
Of course, you cannot conclude from this that “planing” works for everybody. In fact, we included a third rider in the tests, who thought he could detect differences, but whose observations were random. So we already know that for relatively small differences in frame tubing, there are some experienced riders who don’t feel the difference. We thought about expanding the study with more subjects, but concluded that it would not tell us anything new.
The number of study subjects would have to be large to get statistically significant results. We’d need 100+ study subjects, and all we’d show is that some people experience a difference in performance and feel on these bikes and others don’t. It might be interesting to compare more dissimilar frames – say a Surly Long-Haul Trucker with a superlight frame (with the weight equalized), but I am afraid that running tests with hundreds of subjects is beyond our budget. And in the end, the readers still would be left wondering whether they belong to the group for whom “planing” matters or to the group for whom it doesn’t.
In summary, we showed that “planing” exists, not how prevalent it is.