I've experienced it. It comes down to one thing: consistent pressure on the ulnar nerve, which runs through the fleshy part of the hand. Saddle tilt should be considered. Handlebar height relative to saddle height, distance from saddle to bars also should be looked into. You might consider playing with an adjustable stem until you get things right.

I find gel pad gloves to make things worse. But I love the gel capsules that go on under the bar tape. Since you're posting in the touring forum, I think we can assume that you're not racing. If your bars are below your saddle height, you're probably best to consider making changes there first.