I just hobble around at stores and whatnot in my cycling shoes. I just got a pair of "touring SPD shoes" that have a recessed cleat and that makes it less noisy. Still a bit awkward but quieter.

I keep a pair of street shoes at my desk to change into, when I get to work.