My riding scenarios are very much like yours and I went with an entry level road bike. It serves me well enough but now having a couple years on it I got a better handle on what I want.
With that in mind and a desire to add some gravel/trail rides, I've decided to switch to a cross with higher grade components and bought a CAADX 105 just a few days ago to be my main ride. It's not in my hands yet so I can't offer you any first hand insight on performance, but just that after going the road bike route, I arrived at the same conclusion that you did.
Last edited by SpecialJ; 05-13-14 at 08:36 AM.
Reason: removed quote