As a native of Southern California who now lives in the mountains (So Cal mountains), I can say you will definitely need a new wardrobe. Southern Californians tend to put on a heavy winter jacket over their t-shirt as soon as the temperature drops below 60. People who live in colder climates wear layers of clothes and the jacket you wear isn't usually that heavy.