Me to, if one is just taking about the major loads, there are other effects in every imagineable direction. Normally a hole is drilled at an angle, and a tube like stainless brakeline is introduced in the top tube and brazen in place, so there shouldn't be any access for anything into the top tube, though it ads complexity and weight for little discernable benefit, but it is a clean look.
Also, I thought they were mainly punched dead center at 3 or 9 oclock, not the underside. Which has nothing to do with where they might best be punched if structure was a major concern.