Do some recovery rides (light, easy spinning on flat ground for 10-30 minutes.) That's much better than taking a day off. You should still take a day off each week, or at least one day off every two weeks, but fit in some easy recovery rides to keep your legs loose and your blood flowing.
Also, make sure you warmup before and cool down after each ride.