You are putting them on before your shorts, right? The shorts add a bit to help hold 'em up, and most shorts are too slick to provide grip if you put them over.
Be sure to pull them up as high as you can.

Also, develop your legs. Large quads are a plus in many ways.