I've only biked in the summer. Going to try to stretch into the fall

Two questions:

I have light short sleeve jerseys which have been fine for summer. I have stuff to put on my arms and legs (e.g. knee warmers, etc.) but I am still thinking the Jersey is VERY thin material. Do you guys layer, or do you step up the thickness of the jersey material? Is there a certain type of Jersey or layer to look for?

Also, the acorns are starting to drop around here in my very wooded section of the northeast -- bike hazard or no? (at high speeds)

At what temps do you find you start putting on more from your summer gear?