The people that are saying it won't make much difference have never tried it. In fact the tires make a huge difference and the GP5000s would give you an entirely different riding experience, much better for paved roads.
I have three nearly identical wheelsets that I can swap between a few different bikes (different handlebar setups). If I'm going to be riding a paved MUP, I'm going to use the skinny tires, not the big heavy MTB tires that I use when riding the dirt/gravel roads. The bike is faster and handles much better. There's tread pattern and rolling resistance, but also a big part of it is weight. Compare the weight of your current tire to a GP5000. I'm guessing the difference is substantial, and you will definitely feel the difference in a lighter tire. They're faster and the bike is more nimble - it's just more fun to ride. We use the MTB tires for most of our riding on the local hardpack/gravel roads. Skinny tires are NOT so much fun there and in fact it's a bit risky. We need the grip and cushioning of the larger knobby MTB tires. When we dive off into a paved subdivision, those tires are heavy and slow.
Tires (and rims, and tubes) make one of the biggest differences in the characteristics of how a bike rides and handles. It's rotating weight, so of course. You can't turn it into a fast road bike, but you can make it lean more in that direction with lighter tires. If your rims aren't too wide and all of the riding will be paved, could you go with 28c?