Always get the smaller sized tubes if you need to share them across tire sizes. Tubes balloon up to the diameter of your thigh easily, but when they're too big for the tire, it's damn near impossible to get the tire to seat right because the tube is in the way.

BMX riders have been known to ride on 16" tubes stretched around their 20" wheels.