Yes, CO2 (a carbon with two oxygens attached by double bonds) is not going to be physically smaller than either the N2 or O2 in normal air. But chemically it behaves quite differently. N2 is essentially inert with respect to the hydrocarbon compounds in your tube rubber, so the nitrogen molecules just bounce off the inside wall of the tube and are unlikely to ever work their way through and escape to the outside air. OTOH, the CO2 molecule with its O=C=O double bonds can temporarily form a chemical attachment to the rubber molecules in the tube and effectively stick to the tube surface. From there some of them can gradually migrate through the tube and diffuse through the material until they emerge at the outer surface and escape into the air. The effect is clearly evident if you keep track of your daily pressure loss after filling with a CO2 cartridge in comparison with using an air pump.