As long as the bolts are tight, 5mm is more than enough. I imagine if you have them cantilevered out to hold up fenders as well as a rack and let them get a little loose 5mm might not be enough. I recommend not letting them get loose.

When I had a rack and fenders on the same eyelet, I ran the bolt from the inside, put the rack on first, used a nut, then a washer then the fenders, then a washer and then another nut. Seemed to work well and there was much less chance the bolt would loosen up and get hammered by the rack.