I think it's important to take time off. I usually take a week off when my season is done, and the next three weeks are "ride for fun" only (which will include some group rides, but if I want to go off the back and be social, I do that). The goal is to be really looking forward to training for next year when the training season starts. I don't want to hammer myself all year, get burnt out and quit. I've already done that once. I made myself hate riding and as a result I was off the bike for 8 years.

I know I am losing fitness but I seem to be able to be fit for the races I care about (I usually aim for a peak in April or early May, and another one in September). Even with the easy month I don't lose that much, and it builds year over year.

Try to make the time off count for something- do all those things you have been putting off because you're busy training. Have some family time. Etc.

To answer the OP's question, I feel funny and go slow the first ride after a rest. I've heard it called being "blocked". I don't know the physiological mechanisim that causes it.