What you are experiencing is very common. There's even a name for it: chondromalacia patellae. I rotate my cleats until my foot angle is parallel to the center line of my bike. You might need clips with a smaller possible angle of rotation. Fix it before it gets worse.