You're absolutely not crazy. I got back on a road bike about 3 years ago and haven't gotten back on a flatbar bike since. In fact I just dropped the bucks to pick up a Giant Anyroad Advanced 1. I chose it mainly because it had a more upright riding position designed in, that was important to me because an old neck injury really doesn't allow me to spend much time in a more aero position.

Find a road bike that is your size and above all get a bike fit. Then ride, ride, ride.