I'm just the opposite. I never really preferred James over Jim until the last few years, but I have ALWAYS hated Jimmy. It always has seemed like such a pejorative to me. I have a few cousins who insist on calling me that and it always annoys the heck out of me