Ride whatever you like the look of so that you're happy to get out and actually ride. Never buy a bike that doesn't float your boat!
I remember, way back when I was interested in them, I was looking at the Cervelo S2 and R3. I remember reading somewhere that despite the non-aero tube profiles of the R3, it had very similar aero figures to the S2.
An aero frame will save you literally a few seconds over a 40km TT when compared to a old school round tubed frame. It's one of the least economical ways to gain time.
I race, and for me, having a frame that has a good solid BB so that when I get up out of the saddle and climb or sprint, I'm not flexing it all over the place. I ride in bunches, never solo off the front and punch for a sprint if I can maintain the pace to be there at the end, so the aero benefits of an aero frame would be wasted on my riding style. An aero frame looks king of cool, but they're traditionally flexy around the BB. Some of the more modern and very expensive frames are way better in that respect apparently, but they're out of my price range, and so not even on the radar.