I'd use wood if I had to, but

a) wood is softer and weaker than aluminium - I'd expect that the bearing surface would get marked/damaged/distorted easily; would have to make (an)other pair(s) of blocks;

b) lubing/friction will be less consistent with wood than aluminium;

c) hole saws do not cut accurate or smooth holes (not even Forstner bits). They are meant for making holes that you put something like electrical wiring or heating pipes through - clearance is the goal, and 1/8 inch bigger than nominal size is no problem; and

d) if you don't get the saw split at exactly 1/2 you will have problems getting one of the block pairs around the tube; better to get two blocks with flat faces and bolt them together before making your hole - the centre-line will help keep the drillbit in the middle. (If you are boring the hole this should not be a problem, but drillbits sometimes go where they want to, not where you want them to.)