It's most likely your handlebar and handlebar position. The level handlebar would help, and a handlebar higher than your saddle is even better. But of course, there's a question of comfort vs performance, so if you don't obsses about every second of your ride, try to get a more ergonomic handlebar.
The most comfortable handlebars are probably in the North Road "family" of bars(Nitto makes North Roads, and Rivendell sells exclusive Albatross, also made by Nitto), but like I said, they are not for the aerodynamic weenies. All are utilitarian uprights, not your typical racing style bars, but you should try out other racing style comfort handlebar if thats a big deal.