*I biked in the Myrtle Beach area of South Carolina. It never really got too cold for me, just annoyingly so. I'm not big on cold weather biking and I found the weather just a bit too cold for fun.
*RonH is right. If you intend on riding frequently, or especially if over a mile, fenders are a must. Nothing says cold like wet.
*What RonH says is good about cotton and layers is good too.
*Freezing rain is another story.
The bike I have now, a racing bike, would be pretty screwed on ice, only certain bikes can really handle an icey road, i.e. a hybrid bike or a mountain bike. The former may not be able to handle it at all.
Freezing rain storms aren't frequent in these areas, but we all hear about on the news how the city gets crippled when an event like that happens. I'm imagining myself using a bike to get to work and not being able to for a few days. In summary, freezing rain can really only be stopped, for sure, by a wider wheeled mountain bike, which even though posts hazards.
My only suggestion would be catering your bicycle to meet an icey road situation be that keeping pulblic transportation or even just walking as an option