I wouldn't cycle in shorts anything below 50 degrees, and even that is probably on the low end of reasonableness.

Why take the risk with your knees?

If you're feeling kinda Homer Simpson though ("Don't you hate pants??"), invest in some leg warmers for those ~50 degree days and quality tights for when its colder.