Generally, you get the same size leg/arm warmers as you wear in shorts/jersey. I wear XL shorts and XL leg-warmers. Try getting the warmers from the same manufacturer as your shorts.

I would not recommend taking too much stock in sizing charts. Use them as a place to begin. In regards to this one, sounds like there's a mis-communique somewhere.