Either too much play up and down (more on the nondrive side and even then the friction has still bad), or too much friction, and the axle would barely spin.
Is it possible you put too many balls on one side and/or not enough on the other? It is a very easy mistake to make...