Went to the LBS and took a longer ride on the L sized Defy (not the exact model I'm looking at but frame is the same). They laser measured me, took the bike out and also tried it in front of them in the trainer. I think it will be just a matter of getting used to it and possibly swapping out the seat but the L seems to be the correct fit. Any other oddities are likely just getting used to the bike coming from a hybrid Specialized Sirrus. Great input and thanks everyone!