I ride bikes with 170 - 175, and the difference isn't noticeable to me, but the difference in the bikes themselves may mask the issue. Also, my ideal size (as I recall) is 172.5, so bumping up or down 2.5 mm is fine with me.

I can definitely sympathize with noglider that sometimes the amount of knee bend is the issue... when my knees are stiff, I know one of my bikes that I converted to rail trail use from a mountain bike is more difficult for me. Probably due to the 175mm cranks. A rider with short legs would just accentuate that.

I think it was the Sheldon Brown site (I read about it somewhere quite a while ago) that riding style also comes into play...

If you are a masher, and tend to ride at a low cadence, then longer cranks can give you additional leverage.

If you are a spinner, and tend to ride at a high cadence, then shorter cranks mean your feet move less per revolution, and you could theoretically spin slightly faster.

So, my opinion, especially since you feel your current cranks are too short is to try it out. You can always swap the cranks later if it is unbearable.