OP,
Take it from a guy that has owned both...me. I am faster on a so called comfort bike. Some won't understand this...in fact many won't.
OP...there is virtually nothing between the bikes in speed. You need to choose the bike that matchs your flexibility. If you are into bike racing and watched the TdF for example, top riders have the flexibility of gymnasts. Of course the general public all want to look and ride like pros and they go out and buy a bike with a short head tube and a low handlebar. They end up defacing the bike with their riding position. They strain in the drops with straight arms because their flexibility doesn't match the height of the bike. I see it all the time and ride with many like this. Almost everybody rides with a lower handlebar than me. I don't get dropped. My body position is many times lower than theirs. Aerodynamics on a roadbike isn't proportional to handlebar height as some would believe.
I believe you can simpify quite easily which bike to purchase. Call it a litmus test. If you can place your palms on the ground lock kneed, then purchase a short head tube so called racing geometry bike. If you can't, purchase a high performance endurance geometry like a Roubaix SL3 Pro or Sworks like I ride or equivalent. You will be faster on a bike that fits you properly and slower on a bike that doesn't. Simple as that really.