Depends on your riding style, flexibility, etc.

I'm between a 52 and a 54, somehow, and am much happier on my 52 Domane than I was on my 54 Defy. (And When I was bike shopping, the 52 Domane was immediately more comfortable than the 54 Domane. So it wasn't just a matter of the different geometries.)

Personally, I would go for smaller size. Easier to adust up than down.