As long as you're open to the idea of something other than Specialized, do yourself a favor and try the Cannondale CAAD10, too. It may not be right for you, but you may love it. It has a very different geometry than the Allez. And keep in mind, while I have no personal experience with it, MANY here have said that higher end aluminum will ride as good or better than low-level carbon.

Speaking of high-end vs. low end, Specialized has two different aluminum allez frames. The lower level is A1 and the higher is E5. The E5 has a VERY good reputation and is available in their top three models.
http://www.specialized.com/us/en/bikes/road/allez

Good luck!