I agree with you, and I have seen various pros in books and stuff, who worked a whole life making fabulous bikes with very minimal tools. The fact we have to "posit" this kind of thing, shows what is really going on, it isn't a hobby. Every person who want to make a wooden cedar strip canoe, can undertake the project, and finish it, and it all works out. The only thing like that here seems to be the wooden, carbon, and bamboo bikes, for various obvious reasons. People show up, say they want to build something, ask a few questions, and then we sometimes see pictures later.
I'll also posit this: companies like Henry James have taken their craft to the level that it is possible to build a safe, reasonably attractive lugged frame without touching a file to the lugs at all. If I really wanted to, I'd wager that I could take a box of True Temper tubing and Henry James lugs and, after mitering the tubing and cleaning everything thoroughly, simply plug it all together and braze it up as it is. The resulting frame wouldn't win any awards at NAHBS, but it would (assuming the builder ensured full penetration and adequate alignment) build into a perfectly safe and competent bicycle.