Sounds like a hassle tangled up with a nuisance. Imagine getting ready to go for a ride while your buddies wait: "let me just get my shoes... oops we're taking the mountan bikes so I have to change cleats... it shouldn't take longer than ten minutes... maybe fifteen if I want the cleats to be properly aligned." It would be easier (but almost as ridiculous) to only own one pair of pedals that is swapped between bikes.

Also, having only road shoes is generally a bad decision - as decisions that take aesthetics before function often are. A quick run to the store for bread and smokes becomes a choice between riding with street shoes on the most uncomfortable possible pedals, or enjoying the ride but dealing with a sidewalk or tiled floor like it is a skating rink.

edit: I should also mention that the threads in cycling shoes don't last forever, and every time you remove and re-tighten the bolts, they get a little closer to failing. SPD cleat threads are more prone to this, although they are often easily replaceable, but it is easier still to avoid tempting fate and get a second pair of shoes for mtbing and errands.

Bad ideal all around.