FWIW, I wear L shirts, sometimes M will work. Depends on the brand and style. I've been wearing bibs since 2004. "Once you try them you'll never go back to shorts."
Bib sizing is based on height more than weight or waist size. I've tried bibs that fit in the waist but were kinda short. So I determined long ago that XL is the best choice for me. I have now or in the past had De Marchi (my favorites), PI, Endura, Etxe Ondo, Voler and one or two others I can't remember. I've tried "cheap" bibs but they never fit or seem comfortable.
Good luck with them.
Oh, and stay away from Sports Authority, Dicks, Wallyworld, etc when get anything bicycle related. Either shop at your LBS or go to one of the many reliable online sites. If you need help finding them let us know. I can give you several dozen.