Ceteris parabis ...

(1) folded size
(2) you would expect smaller wheels to have more resistance
(3) the smaller wheel will lower geometric trail
(4) you have greater tire/rim/spoke selection -- especially nice ones -- in 406/20".
(5) gearing is lower with 16" wheels.

Now you can compensate for these things but often run into limits in one way or another.