Definately sounds like a bike fit issue. Remember that a bike that's well suited for races and aggressive short rides probably won't also be comfortable for centuries.

How much of a drop do you have from your saddle to your handle bars? A drop of more than a couple inches is nice and aerodynamic for races and short rides, but will put a huge strain on your neck/shoulders/hands/butt/back during a century attempt. A bike that's comfortable on long distance rides will generally have a longer top tube and higher handlebars, keeping you somewhat more upright and reducing strain on the hands and neck.