The negative side effects include less traction on gravel, less bank angle before you snag a pedal on the pavement (because the small tires put the bike lower to the ground), and less resistance to pinch flats (although you compensate somewhat with higher inflation pressure).