Every other controllable variable being equal, which wheel has less rolling resistance? The smaller wheel, since it encounters trivially less atmospheric friction.
The larger wheel, however, falls trivially less far into depressions and trivially better irons out surface irregularities, which exactly compensates for its greater rolling resistance.
The effort, measured in time and expense, to replace one wheel size with another (every other controllable variable being equal) will put you trivially closer to the grave.
Better to go out for a lovely ride this afternoon on the wheels you already have.
Correct. The difference between 559mm rims and 622mm rims with the same width of rim/tire is tiny. That much of a difference might make you want to use slightly different gearing to compensate but other than that, it is indeed trivial.