>> But every once in awhile I would hit a pothole or have to hop a curb and it would fall off. One of those times, the s-hook got caught in my spokes in traffic and I almost went down.

Sounds like you are using an old fashioned mounting system. Modern clip-on systems like Ortleib and Carradice (the Rixen and Kaul Klickfix system) are much quicket to get on and off and cannot be dislodged.
I used to use a hook and elastic and suffered similar problems, but since switching to a modern system I have riddden with bigger loads on bumpier tracks at higher speeds and never have a problem.