Yeah, the tube would have to be thinner or use shorter butting or something like that if the larger one is the same weight as the smaller one. I suspect it isn't though.

Stiffness is mostly a resultant of cross-sectional area to length, so the bigger the tube the stiffer it is. Wall thickness does play a part, but only a minor one.