Feel the cable-tension between the shifter and RD body. It should be taut when in it's most relaxed gear - the smallest cog on the cassette. Not TIGHT - but taut. Not being to see your clockwise, right, counter-clockwise, etc. from here - twist it 1/2 turn and feel the cable. Is it tighter? Or looser?
You may have a bit too much tension or too much friction in the cable housings. If you continue having sluggish shifting you may need new cables and cable housings. If the shifters are more than a year old you may need to shoot WD40 into the shifters.