My personal experience is that cycling specific clothing is better than street cloths if you're going for a workout or extended ride. It wicks away sweat better, cools better, prevents chafing better and I like multi layers since I can add and remove to dial in my comfort.
But for social and around town street cloths are fine.
I once went out for a 6 miles slow ride with my wife and wore street cloths with the exception of clipless shoes for my cleats... I was uncomfortable in the bum.
Having cotton briefs and shorts was no bueno on this ass hatchet.