The seal should be independent of the bearing balls. Bearing contact is steel balls between 2 steel races. Seal fills gap between outer parts of races to keep crap from getting into bearing contact area. Going sans retainers just means more loose balls, not too many though. Without seeing your actual parts I could be wrong. Possible your headset may use the ‘squarish bottom’ of the retainer as support for a seal, can’t tell from here, wouldn’t think so.