I'm 6'1 with a 35" inseam. I have a 58cm 2010 CAAD9. I had a $250 Retul Fit and the 58cm was spot-on for me although my fitter said I could also have fit into a 60cm without problems.
I think the 54cm is too small. 56cm at a minimum.
Oh, and I don't know what I'm talking about so take it with a grain of salt. Go see a professional and ride the bike.