I race on a CAAD9 and train on a Specialized S Works E5, like you I lust after nice stiff aluminum frames. I use aluminum bars, stems and seatposts as well. I've ridden all day on both frames, no problems, not uncomfortable at all. If the bike fits then it will be comfortable, is it a harsher ride then carbon, yes but I like the feel of aluminum.

Setup both bikes the same and you should be OK.