When I got a new mountain bike last year, it came with 170mm cranks. My previous mountain bike had 175mm cranks, and I've gotta say it felt like a big difference at first. So much so that I replaced the crankarms on the new bike for a pair of 175s I found on eBay.
Given time, I probably would have got used to the 170s pretty quickly, but I chose to keep mountain biking with 175s.
The weird thing is, I spend more time on my road bike than my mountain bike, and my road bike has (and has always had) 170mm arms. So even though I spent more time overall on 170s, that length felt really "off" whenever I hit the trails.
So in summary... Do I notice a difference in crank lengths? Yes. Do I think it would make a difference in the long run? Not really (for me, anyway).