I have the same issue, and it's certainly related to my riding position. When I get lower, flatten my back, and bend my elbows more, it gets better. But as the ride progresses, the conscious effort to keep in this position lapses.