If we quit riding at 50F, we'd miss have the season here in Minnesota. I'll ride down into the 20's but I tend to stop when the ground freezes since I ride at night. When the urge strikes me I put the studded tires on and ride whenever. Generally, when it's in colder than freezing, I'm skiing anyhow.
There are more here much tougher than I. One night I was driving home from a work in a blizzard and stuck in city traffic. Some intrepid cyclist kept catching and passing me at the lights despite 8" of snow on the ground.
J.