The biggest difference will be in the ride quality. 25c are much, much smoother riding than 23c. The rolling resistance difference is negligible, if any.

The downside to 25c? Having to re-adjust your brakes to get wheels off and after you put them back on.