i've never had any problem just pulling them off with pliers, nor have they ever accidentally come off. I've reused them many times. I tend to just lightly tap a suitably-sized nail into the opening. Works fine.
I would never pay for them. The LBSs I go to have just given me as many as I want for free. Housing caps too (even in a really sharp black). Take your wrench some beer every now and then, and you'll be repaid a lot more in the end.