What Grambo said. Depends what you mean by 'lowest gear or two'. This is often hard to understand on forums.
Upshifting is technically stair stepping Down the cassette. Are you speaking of lazy shifting to the smallest cogs?
This is generally due to cable friction as Grambo said. The way a derailleur works is the rotory spring rate is most challenged on the outboard i.e. smallest cogs because the spring is less coiled within the derailleur. So generally increased cable friction due to worn/dirty cables will affect outboard rear shifting the most. A tip is...if you leave your bike for a long time...generally shift it to its smallest cog which relaxes the derailleur spring the most. Same with front derailleur but a bit less of an issue.