I am not high on that Crow Canyon/Norris Canyon loop. Too much traffic, not enough pleasant. Norris Canyon is good, the rest, not so good IMO.

If you're looking at that type of distance, there is a good lollipop out of Castro Valley, go Redwood to the south end of Pinehurst, go the length of Pinehurst to the north end at Skyline (you have a short climb, a descent, and a climb), then left on Skyline, left to stay on Skyline, left to get back on Redwood, after a couple of miles you pass the south end of Pinehurst and are back on the stick of the lollipop, continue back to Castro Valley. Length depends on where in Castro Valley you start, but roughly 35 miles, and some decent climbing, the toughest being the last mile and a half up Pinehurst to Skyline. Not traffic free, but not bad, either (and Pinehurst is very lightly traveled). Plenty of bike traffic, so drivers shouldn't be shocked that you are around.

As cthenn has pointed out, there are a lot of great rides from Castro Valley. One reason to take Crow Canyon in spite of the traffic is to get over to Diablo from CV. As cthenn also pointed out, while you can go south, Niles Canyon is scary as heck due to it being narrow in too many places with cars driving fast and too many of them like idiots. Palomares is a great ride, but I'd almost always do it as an out and back just to avoid Niles. If you want to go south, perhaps head east over the Dublin Canyon and the south on Foothill in Pleasanton, and then you can go out Calaveras, or take a nice ride up the dead end of Kilkare Rd. Or if you are bold, make a loop with Palomares using Niles Canyon, but it is scary. Going east, there is a decent loop going out Dublin Canyon, cut to the north side of 580 at Rowell Ranch, work your way through Dublin/San Ramon to Tassajara Road, over to Highland, Collier Canyon, Portola to Isabel, Jack London/Stoneridge back to Foothill and over to Dublin Canyon, that is probably just under 50 miles, depending on where you start in CV, all a loop except once you are east of Rowell Ranch. Perhaps not enough non-urban (and I am not a fan of Dublin Canyon except as a utility road), but a nice ride. Plenty of good riding in the hills north/east/south of Livermore, and if you are sick of Dublin Canyon, you can always take BART get out to Pleasanton/Dublin faster.

If you want a shorter ride, Cull Canyon is a decent out and back, a dead end canyon so not a lot of traffic once you are on Cull Canyon, a fairly moderate climb out to the end. If you are really adventurous, during "Open Season" you can take a visit to the nudist club at the end of Cull Canyon, they let you visit up to 3 times before you have to decide if you want to join. It doesn't matter if you're in cycling gear because you aren't going to wear it anyway. I have never been there myself, but 30 years ago I knew a cyclist who was a member.

I once lived in CV, and long after I moved out, for a few years I went on rides with a shop out of CV before it closed, there are a lot of great rides you can get started from there, from short to really long. There are many good places in the Bay Area to be headquartered for cycling, CV is one of them.