If you switch to narrower tires that are otherwise identical, you are not likely to notice a difference in rolling resistance, and if you did it would likely be an increase in rolling resistance. You are more likely to notice a weight difference when going uphill, and perhaps a small difference in acceleration.
To decrease rolling resistance you need to get a tire with a more supple casing, like switching from a flat-proof commuter tire to a lightweight racing tire. Tires with a more supple casing, however, are generally more prone to punctures... so even if your new tires save you 2 minutes over 40 km, if you have to stop and repair one puncture that you otherwise would not have had, the advantage is lost.