I did the sissy thing & put gel inserts under my bar wrap.
Now, if I wear gloves, it's only on long rides or when I'm going to be sweating so they keep my hands from slipping.
I've been riding for 32 years... no nerve damage here. I think you'd feel some indicators of impending nerve damage if you were at risk.