If you are only wearing a jersey then it is functioning like a base layer - transporting moisture away from the body. If it's hot you don't need an insulating layer. Instead, you want clothing that will promote the body's natural evaporative cooling process which is what a close fitting jersey will do. When it's cold, the base layer does the same thing - moves moisture away - but now you want to hold onto the heat so that's what the insulating layer is for.

What some people don't realize is that a base layer can actually keep you cooler on hot days by enhancing the evaporative cooling process. My merino base layer will get damp and as air flows through my jersey it helps to cool.