This is a tired argument. Obviously the rider is the most important factor in determining speed. But tires can and do influence speed in a variety of ways. The most obvious benefit of narrower tires is the increased aerodynamics. But, depending on the tire pressure, weight of the rider, road conditions, and other aspects of the tire's construction, a narrow tire may actually not roll as well as a wide tire. Its difficult to say without more information. In the OPs case, I'm betting the 7mm difference between 25s and 32s is not going to make a real difference. However, depending on what tires are presently on the OPs bike, a pair of higher quality tires, even of the same width, may significantly improve performance.