Well...
-If you're riding on the roads (even crappy ones) or other improved surface, they aren't needed.
-They rob power while pedaling, so you don't go as fast.
-Added weight, so again slows you down- especially when climbing.
-One more moving part to maintain.