It's a matter of a personal preference. There is no rule. Only discomfort, which is a relative term.

I've done a bit of touring on 28 mm wide tires (700x28c) and a bit - on 42 mm tires. I prefer the latter, and am not going back. Had to build a set of my own 650b wheels for the purpose, totally worth it!