If you go too small, it can affect handling. Changing from 2.5" to 1" will alter the trail of the bike and may result in it being susceptible to shimmy at high speeds.
I've been down the route of getting the narrowest slicks I could on my 26" bike. In the end I found tyre construction to be the most important factor in reducing rolling resistance (mostly light flexible sidewalls).