I commute every day on a singlespeed, and my office is on the exact opposite side of nob hill from me. Since its fairly easy to get around the hill (skirting it to the south past union square), I have no problems. If I lived at the top of a hill, it would be a different story.
That said, I'm building an IGH wheel b/c I want more access to parts of the city that are off-limits on a SS.