Pretty much, yeah.

I convert fixies, but never see a need to file things away. Who cares if it has the marks of previously being used in another manner.

Soonerbills, I'd venture to say that the rider you witnessed and similar ones of his ilk wouldn't do any better on something with gears and everything else. How many drivers do you see in Porsches or other sportscars who've never had a proper driving lesson? What about motorcycle riders? The problem you mention isn't a result of the tool being used, it's a result of the active ignorance of the rider. Demonizing a type of bicycle won't get people like that to stop making ill-informed decisions.

FWIW, I'm not the most advanced rider, but I do find riding a fixed (with brakes) in the city gives me more control in driving; one thing, being able to make rapid sharp turns around cars and traffic jams can be really helpful. I also tend to pay a lot more attention to the potential for getting doored, which is a constant concern.