I prefer to go faster
I'm actually more nervous descending if I'm going slower than traffic. I feel like that confines me to just the shoulder of the road, while going faster enables me to take the entire lane. The key really is to look ahead of you, and don't have a death grip on the bike. Never turn the handlebars. Just use your body weight to flick the bike around holes and cracks. You'll get use to it after a while.