I'm having this debate myself. I'm thinking of going down to a 28 or 25 from a 32, and I haven't decided between 4 Seasons, Gatorskins, and GP4000s (or something wild like going tubeless).

In terms of rolling resistance, the Gatorskins have roughly twice the resistance depending on PSI compared to GP4000 RSs and about 15% more than the 4 Season (22W @ 80 PSI vs 12.3W and 19.8W, respectively). The GP4000S II is 13.7W @ 80 PSI, for what it's worth.

My commute has some pretty crappy spots with lots of gravel and debris, so I'm tilting towards something tougher, but the temptation to pick something faster is there for sure...