Well, wider tires have less rolling resistance and they are more comfortable so I would go with the biggest size you can stuff in there as well as fenders.

That said, my commuter runs 28mm with fenders. I would go wider if I could.

Look at some of those cyclists rolling on 42mm 650B hetres. Some of them still blaze. A lot of tires 32mm and up are lower quality, hybrid tires, which can account for the sluggishness in changing sizes. Go with a high quality tire (or any tire that is comparable to the 23mm you are riding) and you will notice an improvement.