that is my suspicion too.
Invert the fork, assemble and test.
Much easier than knocking them out.
Be sure to check the cups and crown race for indentations.