It's not the adjustment, it's the cabling. Either the housing is crudded up, or the cable is cruded up, or the cable is attached wrong, or something is bumping the cable. When you downshift, check to see where the tension in your cable is. Somewhere between the place where there is tension in your cable and where the cable is slack and loose is where the problem is.