My top speed has been 50 MPH, which I have done a few times. I have been afraid of the situation and slowed down even at 20 MPH. It all has to do with the situation.
Something that slows me down almost as quicikly as a rough road is the wind. A cross wind during a descent can be brutal. Our highest peak is Mt. Magazine. During a descent on the North side I found anything over 20 MPH was scary in 30 MPH cross winds.