It sounds like a cable issue to me. Any basic bike will come equipped with low-grade housings and standard cables, which IME are good for a year or so. Go for a better quality cable (Shimano Dura Ace is nice, but $19) and good quality lined housing (Jagwire, etc) and make sure you're fully housed from shifter to the rear hub arm.
This is a starting point.
IME the grip shifter and trigger shifters for Nexus and Alfine are pretty reliable, and the hubs themselves are super reliable (unless you have a faulty one). Cables have been the biggest factor in shifting performance from my experience with them.
The bumpiness should not be an issue. I've ridden Alfines offroad with no problems, and many MTB'ers ride these as well (mainly in Europe).

If it is "hanging up" or delaying in its shifting, I'd say 1. cable or 2. faulty hub (including incorrect lubrication or bad spring, etc)