The best article of apparel I have ever purchased for cycling in cool or cold weather was my wind jacket with removable sleeves. A cool or cold wind will cut through a long sleeve jersey and perhaps another layer. When you perceive the temperature to be cold enough, dress in layers as others have mentioned, and use the wind jacket as the outer layer. Someone else mentioned that you have to experiment to determine what will work for you, and I agree. Also, I agree with starting "chilly". Usually, I'll begin to warm after a few chilly miles. Get a wind jacket and enjoy the riding and experimentation.