I ride a fixed gear all year for commuting, my commute is about 26 miles RT and I also use it to get around the urban sprawl. I've never found an issue with using a fixed gear in the city as long as you're not crazy and use a front brake.

Where the real pain is is jumping back on a freewheel/freehub style and thinking your chain is broken.