Ride with what's comfortable for you. Ideally you should be on the road, but if that scares the bejeesus out of you, then it's not worth the anxiety. Just slow down and watch out for pedestrians. If you're puttering along slow enough on the sidewalk, it is safer.