No, I ride heavily both df and recumbent. I'm also a bodybuilder and have massive treek trunk legs with lots of power. It's simply the less aerodynamic position of the DF on flats and downhills. The DF is significantly faster on uphills, but it's just not as fast anywhere else. It's also nowhere close to as much fun!
Just wish they were allowed in races with DF bikes.