Dont wear a waterproof outer if its not raining. None of them are breathable enough for riding in the dry. You can get effective windproofs which are quite cheap. They are far more breathable than any waterproof, and will allow dampness to evaporate.
Keep a waterproof in yor bag for when it starts raining.
You should start the ride a bit cold, then work up to your operating temp. In cold weather there is no need to dress up so you sweat.
If you do want to wear something extra at the start, check out a sleeveless "gillet". You can get them in fleece, with windproof layers etc. Wear it over your windproof, and take it off when you get warm.