When I'm on my way home through downtown in the evening, it feels safest when traffic slows down at least to the low 20s. That way, I can just ride in the middle of the lane and not feel like cars are trying to squeeze me over into the door zone. If I can keep within 2 or 3 car lengths of the car ahead, the driver behind me usually doesn't seem to get impatient (i.e., they don't honk, tailgate, or cross over the double yellow to pass me). It feels pretty good to move at the 20 mph or so necessary to do this (that's about my limit on the flat, and I can't keep it up for very long, but feels good while it lasts).

In general, I think it's ok to go fast in urban environments if you keep within the limits of your skill and what traffic conditions allow you. It's also possible to use it as interval training. Go fast, but slow or stop whenever needed to be safe. That just becomes an opportunity to accelerate again....