Originally Posted by
chaadster
Can you point me towards a lumen test that compares claimed lumens?
The only one I've ever seen was on MTBR, but they seem to have taken that down, and now only publish measured lux. There's a video showing how they got measured lumens (with the collecting sphere), but again, I can't seem to find those results published anywhere on the site, even though I swear I saw them pop up a couple of, or three, weeks ago.
Maybe they found there was a critical flaw in their test; the collecting sphere did seem small to me, but I don't know much about these things, and can only speculate as to why MTBR took the results down in any case.
It does bring up the question, however, of what type of lumen measurement testing is being done when we see manufacturer claimed lumens. For example, are they actually reporting the headlight itself, or just the emitter under some other operating circumstances. Temperature, current, and placement of the light source in the collecting sphere are all variables that need accounting for before lumens can be effectively compared.
Of course, that may be the very reason MTBR dropped their lumen measurements; they simply were not indicative of anything useful. Certainly lux, if measured properly, as an intensity of light at a given point, could be more helpful to the consumer, particularly because theirs is the only measurement, and avoids the messy aforementioned business of lumen comparisons and of course, gives some measure of actual usable light output at a given distance.
So, I've heard people talk about "lumen exaggeration," but where is the proof (data) and even then, without knowing where those lumens are coming from (actual lamp assembly or emitter) and how they're measured, we can't really make any meaningful statements such as some are overrated.
Certainly standardized testing and measurement protocols for the actual headlight manufacturers is needed, because MTBR can't test them all!
Most manufacturers do claim XXXX lumen output. I believe, as you appear to also, that they aren't reporting the actual output but the ideal output based on manufacturer specifications which are probably based on calculated rather than actual measured output. The 'out the front' light is probably lower than the calculated amount but, because nearly everyone lists the same way, valid comparisons can be made as long as you don't get too hung up on the actual numbers.
The MTBR light measurement method that I have seen is to sit the light on a bench pointing towards a ceiling with a light meter next to the light, then measure the lux from the reflected light of the ceiling. While this probably isn't the best method, at least it's a consistent measurement. I wouldn't say that you or I could reproduce the measurement but for a comparison within their results, it's valid.
We also have to take into account the construction of the lamp. A Magicshine might be putting out the same lumens as the Cree I bought this summer but the Cree has a narrower beam. The narrow beam makes the perceived light brighter because the lux is higher. We really should report lux but even that has its own problems. Without specifying the distance from the source, the lux is meaningless for comparison.