I'm all for saving weight, but if Mavic says don't use anything smaller than a 28mm, I wouldn't put a 23mm tire on it. Are you sure the tire is 23mm and not just says it is a 23mm? Even if the tire would fit, I wouldn't want to tour on such a slim tire. I would only want to carry things on my bike that I could use. I'd rather carry a heavier spare tire that I can put on the bike and not worry about finding a shop or hoping the tire doesn't come off on a descent. Once the spare is on, I can happily be on my way.
Since I have never tried to go smaller than the recommended size, I can't say it won't work. Most likely Mavic's recommendation has some lee way for legality reasons, so it might. I wouldn't risk it.