If the chain is a protruding pin model, you never completely remove the pin. You push it most of the way out but leave one end engaged with the outer plate, so it goes right back in.

If you have a flush pin chain, it should never be rejoined with the same pin. Shimano chains use a replacement pin. I'd use a properly fitting master link to rejoin a chain or buy a brand that comes with a master link.