A tube inflated with CO2 will definitely be softer the next morning than one that was inflated using an air pump. That's an easily observable fact.

For that reason alone, I don't worry about what the CO2 might be doing to my tube because it doesn't stick around for very long.