I have gloves, but usually forget to wear them. I cannot say I notice any difference whether I wear them or forget.

The important thing for me, to avoid hand pain, is to keep my hands moving. Rear shifter is on the right bar end, front shifter is on the down tube, the brake levers are out front where you'd expect them to be, and snacks are in the handlebar bag. My hands never settle in one place for long.