I'm also in Rochester. I live on the east side and work in Gates, so I ride through the city. My experiences are somewhere between yours and tsl's. I had to switch to Armadillo tires due to the debris in the road. Most of the drivers are reasonably attentive and cooperative. But if two or three drivers out of the hundred you see every morning are jerks, it seems daunting.

Vary your ride, you'll find some roads are better than others, in terms of both road quality and driver quality. Where do you ride?