Wood and metal hardnesses are tested differently, so comparisons are not simple or easy, but back-of-the-envelope-type artithmetic says that 606T6 is roughly an order of magnitude harder than the hardest wood known. Friction and lubrication may be inconsistent because of grain and pores. And the problem of the saw kerf is that if you go off the centre-line either way you can't get one of the halves around the tube.

I argree (partly) about equipment, but drills and drill presses are for holes, not bores - cheap drill-press runout can easily be .1mm. Sometimes accuracy is the result of luck.