I'm exactly the same size and also wear a medium. I find that there's a "US medium" which would be considered large anywhere else. Brand like Voler or Garneau sell medium stuff that would honestly fit someone who's 6'1" / 180 lbs. In contrast, European brands like Castelli and Giordana fit me like a glove. I find that if I go online and a bunch of Americans are complaining that "the medium is ridiculously tiny!" / "I couldn't get the bibs past my knees!" / "this is sized for midgets!"... it's going to be perfect for me.