A year or two ago there was a chorus of voices suggesting people (everyone?) switch their 27" wheels to 700c. I raised the point at the time that it would reduce the trail by about 1.16mm, and asked whether this would affect handling. The chorused answer at that time was no, it wouldn't make much difference. Since both A and B were true, where A = enough people say it, and B = I read it on the Internet, then it must be true, right?

So carrying this thought forward, a switch from 32mm to 28mm tires means a cross-sectional radius difference of only 2mm. (Does this really mean a change in ride height by only 2mm, or something different?) That's less than the supposedly insignificant 630 to 622 wheel switch. A smaller front tire means the bike will point slightly downhill but much less than 1deg. This will further reduce trail, but is a tiny amount by itself.

The point being that you probably wouldn't notice much difference. But that's only because the Internet and enough people said so.