Bib and jersey for anything over 10 miles, and I keep a small pair of mesh shorts in an easily accessible pocket for walking around.
Some people are fine wearing whatever for 100 miles plus. But, I hate loose clothes flapping in the wind, restrictive shorts, and am so used to wearing bike shorts/bib that not wear feels super weird. I kinda feel like a tool at times, especially rolling up to some hipster/hippie doods in jean cut offs and a cotten tees, but whateves, my boys feels comfy in lycra/spandex.