I wear shorts along with long sleeve and more insulating top down to around 45 degrees. I just find that the legs which are producing the most heat are better in shorts.

For below 45, I've been wearing pants made out of Event fabric (waterproof breathable), but have been considering getting something more flexible and breathable so I've been looking at Windstopper stuff. Like here's one pair of their pants:
Men's COUNTDOWN WINDSTOPPERŽ Soft Shell Pants | GORE BIKE WEARŽ

Or here's another one:
Men's FUSION 2.0 WINDSTOPPERŽ Soft Shell Pants | GORE BIKE WEARŽ