You could always try an experiment to confirm or rule out your theory. On a cool morning when the tire looks normal, check the clearance then let about 5 pounds out (since you're concerned about the rubber not the pressure) and use a hair dryer or heat gun on the tire. See if the shape has changed enough to make it rub.

I'd advise using some caution with this experiment, monitoring the temperature of the tire, rim etc to avoid damaging something or a potentially dangerous blowout.