The same goes for your indoor environment. A while back, I turned my winter thermostat to 66F in the day and 62F overnight. At first, we were a little chilly -- had to get out the sweaters -- but nowadays we seem used to it. I actually find the temperature at work stifling...
For outdoor cycling, you adjust as the weather gets colder... as long as you don't have a hiatus of more than a day or two.