Either your chainring is out of round, your bb spindle is imprecisely milled or your crank spider is a bit out. Probably a bit of all three. Just about every drivetrain component made will have this issue, and you only really notice when you go to singlespeed and dont have the derailer taking up slack. You can try and minimize the problem by rotating the ring on the spider or the crank on the bb spindle, but the only sure fire solution is to buy very expensive, high precision track parts like Sugino 75 bb, 75 cranks and zen ring. For most people this is unnecessary though ---just do the rotatey trick to try and minimize the problem, then set chain tension so it's good with the cranks at the tightest spot, and live with the variation.