Hard to believe the vise and socket didn't work for you; I can't remember that ever letting me down.

Hammer and punch does work but, as has been pointed, out you will destroy the cotter pin in the process.