As long as you can get the seat up and the bar low and into a good riding position, there is no reason why a flat bar would cause a road bike to lose much speed. If you look at the results of pro gravel races, there are guys who finish 100 to 200 mile long gravel races with an average speed over 20mph with flat bars, and in a much shorter race they would surely tack a few more mph on. I have ridden flat bar bikes with fat tires to around 20mph averages over 20 kilometer distances in the not too distant past. Fast is a subjective term though........