Interesting derailleur question:
My rear 10-speed Ultegra derailleur will not shift well to the rear small cogs. Around the fifth cog it quits wanting to shift to cogs 6-10 (this happens also with the chain removed from the bike). Once I get the derailleur to the smallest cog, I can shift all day long to the next smallest cog and then back again, but once I move two or three cogs larger the derailleur just does not want to shift. I even tried taking the derailleur off the bike and gave it a thorough cleaning, but it still has the same response. Any ideas on what the problem could be? The derailleur is a year old and only has about 4,000 miles on it.
With the chain off does the derailler have a full in and out range of motion? That is, can you push on the lower pivot point where the cage is and push the derailler in so that it is in line with the largest cog? Does it snap back to alignment with the smallest cog when released? If so then the der is not the problem. Next pull the cable and examine it full length as per hill riders suggestion and look at the housing as well for kinks or obvious problems. The cable should move freely in the housing. Finally does the cable move, or wind up when you shift to higher gears. Shimano 8-9spd brifters tend to slip a cog and the ratchet mechanism fails at some arbitrary point in my experience from 2k to 10kmi age. 10spds are per Sheldon Brown more reliable but ratchet failure would account for this problem as well.
Addendum: an additional check on der function is to pull the rear der cable on the down tube away from the down tube while turning the cranks. If the r der is ok pulling the cable out from the tube will cause the rear der to shift onto progressively larger casssette cogs as you increase the pull. You can shift over the entire cassette this way.
as said earlier, sounds like the cable and rear section of housing are dirty. replace both. it could also be that the back section of housing is too short, but it sounds like it was working fine at some point. if that doesn't solve it...
...do you drink any kind of sports drink while on the bike? anything that would dry sticky when spilled on the bike? that crap, along with road dirt/grime tends to collect in the cable guide under the BB. you can either clean that off, or replace it. to prevent the problem from happening in the future, get a couple short lengths of inner cable housing and run it through the grooves and run the cables through that. it'll be smoother for longer.
another indicator for the BB cable guide being crap-encrusted would be if your FD also shifts like butt/feels sticky or rough.
4000 miles is a lot of miles to put on the bike in 1 year, are you racing or touring? if so, bikes used for those purposes tend to get a more worked than others, i would step up your maintenance regime considerably if you're having these kinds of issues.
I had the LBS replace the cable and housing in October (at about 3,400 miles) so it did not occur to me that the housing could be all cruddy again (that is what they said). The bike is a Kestrel Talon and all the cabling is either in housing or inside the frame. It sounds like the cabling/housing might me yucky again so I guess I will try replacing it - it just seems like under 1,000 miles and only 6 months is way too soon to have this issue again.
If the cable isn't frayed, it sounds like a kink in the cable near a cable stop, i.e. it becomes stiff when the kink tries to enter a section of housing. With Shimano rear derailleurs, it doesn't take much to screw up the shifting : the derailleur's spring is often too weak IMHO, and the slightest imperfection with the cable routing or the housing causes problems. Often when I redo a derailleur line, shifting works perfectly for a while, then starts missing shifts once in a while at the first rain shower, and it goes down from there. That's how sensitive it is.
If I were you, I'd buy high quality teflon coated housing and cable, and I'd redo the entire line myself. Unless you know the LBS very well, he probably won't do it juuuust right the way you want it. When you redo the cabling, cut the sections of housing so they make nice "natural" curves between the stops, with the cable entering and leaving the stops tangential to the housing whenever possible. Also, cut the housing with a quality tool and make sure you dremel the ends of the housing flat with the housing arched like it will be when installed. After you prepare an end, don't forget to open up the hole with a ballpoint pen or something, so nothing rubs. Also try to use metal ferrules instead of plastic ones. Then when you slide the cable in the new housing, watch for any sudden stiffness, which would indicate a kink in the housing. The idea of all this is to eliminate friction and slop as much as possible, so the shifting is light and crisp.
If you keep having problems, you can try a Rollamajig (http://www.mtbr.com/reviews/Extras/product_20952.shtml), and if you're still not satisfied and you're a bit of a perfectionist, replacing the derailleur's return spring with a stronger one usually solves the problem once and for all, although it's a real PITA, and you need to know how to make springs if you can't find the perfect replacement.