The riders who are after performance and speed will go for the thinnest tires they can find and they insist the tires make a huge difference. They'll try for 25 mm or smaller.
My riding is primarily commuting, utility riding in town and touring, so my needs are not the same as those of the performance riders. Right now I've got 32 mm tires on my road bike and I won't go smaller than 28 mm. The reason is the riding surfaces. Some of the roads in town are nothing more than collections of patches — the result of years of cost-cutting measures by the municipality. Narrow tires on those roads will be uncomfortable at best. Also, some of my favourite roads in town and nearby are gravel and dirt. Such surfaces are not good friends with narrow tires.
I'm now planning a tour for 2011 which will include several hundred kilometres of gravel and dirt. For that tour, I'm planning on 40 mm tires.