If you were playing with the shifter and loosening the cable, then I suggest you loosen the cable and let the derailer fall into the smallest gear; then make sure your shifter is also on the lowest gear position.

If you had tightened the cable while the shifter was in a high gear setting or if the derailer itself was in a high gear then that will mess up your shifting position.
It's either one of those because your cable will be too tight and be on a middle gear instead of being on the smaller.
Youtube how to tune a back derailer/gears.
That may not be your problem but I've seen plenty of people come to me with tightened derailers that supposedly are on low gear.
Sounds like you have a cable issue and that your cable needs to be in exactly the right position, damn anal retentive derailers.

Either that or try using the fine adjustment screws on it.

ALSO, if you have an old bike, your spring inside the derailer itself might be worn out. I just worked on a lady's bike today. I saw the spring inside the derailer all rusty and it would not compress down to where there were no gaps in between rings even with completly loose cable. Poor spring was heaved and I could not for the life of me, tune the back derailer. (But then again, it was a Mongoose. lol)
Might be your problem too.