The right answer is, "Get a smart charger"

You want to charge it until it's full. So how long to charge depends on how full it is to start with, how much current your charger delivers and at what voltage, the uptake capacity of your battery (which is not constant, it decreases as the battery reaches full charge and varies depending on ambient temperature).

In fact, I think in most cases it IS actually impossible for a dumb charger to charge a battery as completely as a smart charger can, even if you get the charge time perfect. Modern smart chargers do an analysis of the battery when they first connect, and they will do pulse conditioning of the battery while it's charging, which can significantly extend both the capacity and the lifetime of the battery. Also at end of charge they'll kick into modes that "fill in the corners" of the battery's capacity.

Basically I assume that if I don't have a smart charger, I'm going to either overcharge or undercharge; getting it perfect is almost impossible. Smart chargers are cheaper than a new battery, so I figure it's worth buying them because otherwise I'll probably forget and leave the thing charging overnight sometime and fry the battery.

I wouldn't even bother buying a rechargeable battery without a smart charger.