what kind of crank and ring do you have? some are rounder than others.

this phenomenon bothered me a whole lot more riding on the street than on the track.