Slicks also won't wear out as fast. The only downside is that once you take this leap, you'll inevitably end up buying a second wheelset for your off-road tires, assuming you ever do that. And I'm not even really sure that's a downside.