New rider here doing 10-15 mile rides in basketball shorts or jeans a couple weeks ago when it was cooler. I do just fine. For now. Give it a try. If I have to do anything different than that in future its going to be covered by something else.

A good 20% of my miles have been riding with my oldest daughter after work close to home I'm wearing jeans golf shirt and full height wellington boots. Her sometimes in high heeled boots.