I do it every single day on my single speed when I ride to school. Adidas Sambas on Look pedals; works fine. The clipless interface side of the pedal provides a bit of grip; it's not all that bad.

I never really think much of it, but then again, my commute's very short (1 mile each way). I wear clipless shoes when going for rides longer than 3-4 miles; normal shoes on clipless pedals gets a bit uncomfortable after a while.

As for damage to the pedals? Yeah, right.