I just went from 60 psi 32c tires to 105 psi 28c tires. The 32c tires may as well have been glued to the forks in comparison. Greatest performance mod you can make IF you are strictly riding in the street. Tread pattern being the same, the laws of physics dictate that you WILL be a difference.
Good for you...
The difference in rolling resistance is negligible. The biggest difference you will notice will be due to rotational inertia which you can achieve by reducing the mass of the wheel and tire. This allows the wheel to accelerate at a higher rate. I do not know the weight of the tires you chose, but even as little as 50 or 100 grams makes a difference in how the bike feels. What is noticeable and what matters are not necessarily the same thing though.