OP, you hit the right idea with rain pants. They serve as great windchill eliminators.
Craft of Sweden makes some nice cool and cold weather tights, with pads. May-June offers great end-of-season deals. Craft o started with nordic-skiing gear. They do a good job with cold-condition high-aerobics gear.
A lot of people like Lou Fox, very reasonably priced , and it's custom fit to you. Lots of wind-blocking, water-resistant, cold-ranges fabric choices.