What's your stock wheelset? I think it's mostly bearing related until you get up above 15mph or so. When I'm riding on some good hubs, I gotta feather the brakes to stay behind lotsa people I encounter on my commute when they start to coast, and I'm not drafting them.