All that said, I agree with the OP qualitatively if not quantitatively. A fan really does help. In winter, I do my roller rides in about a 50 degree room. If I didn't have a fan, it would be just awful. I leave a puddle on the floor even with a fan. I do notice a HR difference of a few beats at the same output between having the fan on and not having it on, not that different than what he has up there. Of course if you only use a PM you'd never see it. Then we could have a discussion about whether or not it mattered . . .
I think heat stress will decrease power output at a particular RPI whether one measures HR or not.