All I can say is that I used to be cold weather averse as well, but less than one year of "gutting it out" in the cold has converted me. Now my main problem is that I'm too warm on my rides - I'm usually unzipping (my jacket, you pervs) in the middle of a ride.
Regardless, do whatever it takes to keep you in the saddle, inside or outside. (I don't want to pi** off my buds over at CycleOps by telling people not to ride on their trainers!)
ps. Hi porkchop...welcome to the forum.