My wife only pulls her knee warmers up a few inches beyond the leg hem of her shorts. Needless to say, they always fall down. lol
I try and try to tell her, but she doesn't listen.
So, my initial thought is that you need to pull them up more. Mine ride up near my groin, and they never come down even a bit.