I leave mine on. I don't know about others, but the hitch pin on my rack (Hollywood Sportrider2) threads into the rack through the hitch (does that make sense?). If you don't screw it in tight, the rack wobbles a lot. I use a nut with Loc-Tite so it would be a pain to remove frequently.

pubb