in general, 23s are narrower, yielding a slightly narrower contact patch. As said, they're "smaller" so there's less air to absorb bumps, creating a stiffer (hence faster) ride.
it's also worth mentioning that the 23 and 25 measures are somewhat arbitrary, and there can be a good bit of variance in the actual inflated width.
I've seen a few "23s" that are almost imperceptibly different from "25s"