Newer derailer, with only 300 miles on it.
Put it on, adjusted, and it was working perfectly up and down the gears. This is 3x7 setup.
Now, there is an issue when changing to 4th gear, and 4th gear only. Going from 3rd or 5th takes much longer than any other. The only way to change it, is either wait, or push the gear selector little further, almost to the next gear. It's quiet when in 4th, no chain rubbing or anything like that.
Another adjusting is fixing the problem, but only for a very short time/distance.
Derailer hanger is straight, new chain installed the same time as derailer and cassette.

I notice it while riding on an unpaved, rough roads, but I had the same issue with previous Shimano LX derailer on the same bike. All cables and housings are also newer. The only things that I didn't replace in between the derailers on the drivetrain, are shifters and the rear wheel.
Bike was never dropped or damaged in any way.

What am I missing? Any tips will be greatly appreciated.