I think it's more mental than anything.
I used to feel the same way you did, especially with anything steep and over 1/2 mile (or say a kilometer for those in the rest of the world).
A couple years ago I started doing hill repeats up a well known stretch of road in my area (leading up to Mt Wachusett) - it never got easier, but I did get faster by finding the right gearing and cadence on different parts of the climb, and was able to find different positions on my bike (moving up in the saddle and lessing my grip on the tops) that lessened the suffering.
On our tuesday night group ride I can now get up the hills at a reasonably fast pace, but I suffer like hell when people decide to drill it on the long flat sections of road.