You may be able to wear shorts in dry cold, but damp air will suck the heat out of you.
If you are cold, wear more, and if you are hot, wear less. Its quite simple.
If you are in the UK check out Ron Hill and Parrot, 2 good value sources of clothes designed for UK conditions.
Polyester leggings, such as Ron Hill are std issue for any outdoor activity that involves cold, rain and mountains.