Joined: Jan 2010
Posts: 9,831
Likes: 1,809
From: Northern California
Bikes: Cheltenham-Pedersen racer, Boulder F/S Paris-Roubaix, Varsity racer, '52 Christophe, '62 Continental, '92 Merckx, '75 Limongi, '76 Presto, '72 Gitane SC, '71 Schwinn SS, etc.
I always use as many as will fit. I've checked many headsets for example where there was a bit of running clearance even after fitting the full complement of balls.
Taking one out could lead to unstable bearing support and localized overloading in cases where the number of balls is small, such as in a hub or bb.