if you're relying on your grip padding to prevent hand fatigue while on downhills, you're doing it wrong.

try not death gripping your bars, moving your brake lever reach closer to the bars, and grabbing/using less/not riding your brakes.