Right. If I put on panniers with heavy stuff in them, the stuff might add enough weight, and the added drag might not slow the bike down as much as the weight speeds it up. On the other hand, changing to heavier tires should not make me descend any faster.

But I'm not so interested in descending. I'm interested in how best to improve speed on flat ground because it factors out terrain. Most of what you can do to get faster on the flat will help for climbing. Improving downhill speed isn't so important. Besides, I'm already a fast descender.