This time of year I ride a lot in Prana Stretch Zion pants.
They are more than flexible enough to ride in, I've gone on 3+ hour rides with them. They have just barely enough room underneath to wear a thinner type knee pad. If you're moving, they're pretty warm on a fall day.
I've actually gone rock climbing in them quite a bit to. Overall, I'm more impressed with their durability than I'd expected.
As a no-butt skinny guy with quads, they fit snug, almost like tights from above the knee up, and very slightly baggy below that. I wear a belt because the waist is also stretchy and eventually gravity overcomes my lack of a booty.
It's absolutely no issue to wear them to the movies or out to dinner. They basically look like chinos.
The baggy ankle is not an issue on a 1x gravel or MTB. It could be if you have a front derailleur.
I just put padded shorts on underneath. It's better than dedicated riding pants.